Page MenuHomePhabricator

Welcome survey: communication and deployment to Vietnamese Wikipedia
Closed, ResolvedPublic

Description

User story
As a new user who registers on Vietnamese Wikipedia, I want to complete the Welcome survey, because this survey is part of how Growth features can be further personalized to address my specific interests

As a product manager, I want the Welcome survey to be part of the "core Growth feature set" that is available on all wikis that have Growth tools, because this data is helpful in understanding each community, and because I want to the Growth team be able to further personalize onboarding based on newcomer interests.

Description

As part of T206365: [EPIC] Growth: Welcome survey the Growth team released the Welcome survey to ALL Wikipedias with Growth tools.
However, there was a concern over the high abandonment rate on Vietnamese wikipedia, so we rolled back. After some initial investigations T216668: Welcome survey: investigate Vietnamese abandonment rate we see:

Acceptance Criteria

  • CRS communicates with viwiki about change and ensures there aren't any objections to this change
  • Engineers enable the Welcome survey at viwiki on January 30.
  • CRS updates documentation: Welcome survey current status updated & Deployment table update
  • PM provides CRS with viwiki welcome survey data after one month (starting March 1)

Separate task:
CRS shares initial welcome survey results with viwiki community: T331566

Details

Due Date
Jan 30 2023, 5:00 PM

Event Timeline

Just noting that CRS will be able to support here after the holidays break - and if anything is instead urgent, you're welcome to reach out to me.

Thanks, @Elitre, this can definitely wait until January.

Trizek-WMF changed the task status from Open to In Progress.Jan 17 2023, 6:10 PM
Trizek-WMF updated the task description. (Show Details)

As the discussion goes on, I think we can set a date for the reactivation of the survey. @KStoller-WMF, can we move it to priorities?

@Trizek-WMF
It's just a config change to enable the survey for viwiki, correct? So it doesn't need to be part of a release train / can happen any day, correct? If that's correct, should we set Monday, Jan 30th as the date to reactivate assuming all goes well with the discussion?

Trizek-WMF subscribed.

It is. Let's start on January 30 then.

Trizek-WMF set Due Date to Jan 30 2023, 5:00 PM.

Sounds good, thanks!
I updated the task description so we don't forget to update documentation: Welcome survey current status updated & Deployment table update

Change 883301 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] Enable WelcomeSurvey at viwiki

https://gerrit.wikimedia.org/r/883301

Change 883301 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable WelcomeSurvey at viwiki

https://gerrit.wikimedia.org/r/883301

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:21:48Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:883301|Enable WelcomeSurvey at viwiki (T325376)]], [[gerrit:884930|Fix grid blowout with limited width turned off (T327423)]], [[gerrit:884929|Support new style of table of contents (T327942)]]

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:23:27Z] <urbanecm@deploy1002> tgr and urbanecm and jdlrobson and legoktm: Backport for [[gerrit:883301|Enable WelcomeSurvey at viwiki (T325376)]], [[gerrit:884930|Fix grid blowout with limited width turned off (T327423)]], [[gerrit:884929|Support new style of table of contents (T327942)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:31:41Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:883301|Enable WelcomeSurvey at viwiki (T325376)]], [[gerrit:884930|Fix grid blowout with limited width turned off (T327423)]], [[gerrit:884929|Support new style of table of contents (T327942)]] (duration: 09m 52s)

KStoller-WMF updated the task description. (Show Details)
KStoller-WMF added a subscriber: Tgr.

Thank you, @Tgr!
I'm now assigning to @Trizek-WMF to complete the next step:

Trizek-WMF updated the task description. (Show Details)

Step completed!

Thanks all. In the future, please make sure that tasks for the CRS teams are separate from others - ideally, children to a parent task.

Thanks all. In the future, please make sure that tasks for the CRS teams are separate from others - ideally, children to a parent task.

Will do, thanks for the reminder!

Etonkovidova subscribed.

Checked on viwiki wmf.24 - the Welcome Survey is present and works as expected.

Moving to Test in Production | Watching, so the following will be addressed:

  • PM provides CRS with viwiki welcome survey data after one month (starting March 1)
  • CRS shares initial welcome survey results with viwiki community

February 2023 Welcome survey data for viwiki (data from Superset):

Response Rate:

  • 1,860 impressions (1,245 on Desktop, 615 on Mobile)
  • 53% complete the survey (47% skip or abandon)

Account Creation reasons:
Of the ~980 respondents that completed this question, these were the reasons named for creating an account:

  • 32% to add a new page
  • 26% to read
  • 15% to edit or change information
  • 12% other reason
  • 10% to fix a typo
  • 3% program participant
  • 2% to add an image

Have you edited Wikipedia before?:

  • 31% I didn't know you could edit Wikipedia
  • 31% "other" or N/A
  • 14% yes, a few times
  • 12% I don't remember
  • 12% yes, many times

Which languages do you speak?

  • 100% vi
  • 44% en or en-gb
  • 3% ja

(and many other languages with less than 20 responses)


My perspective: I don't see any data that makes me concerned with the release of the Welcome survey to Vietnamese Wikipedia. The completion rate is 53%, which isn't too different from the 55% completion rate across all wikis (over the same period of time).
Hopefully this data is helpful for the Vietnamese Wikipedians interested in new editor motivations! I encourage any viwiki community members to reach out the Growth team if there are any questions about this data.

KStoller-WMF updated the task description. (Show Details)

By "47% skip or abandon", do you mean they don't fill out the survey but start editing? Or do they actually abandon their path and never return? It is just a clarification of the wording.

"Abandon" doesn't necessarily mean bounce or leave Wikipedia, my understanding is that it's basically the catch-all for all actions other than clicking "Finish" or "Skip this survey".

When the user sees the survey, they have a series of questions to fill out, all of which are optional. They can then submit the survey by clicking "Finish" (even if they have not responded to any questions), discard their responses by clicking "Skip this survey", or take some other action that leaves the page or site, such as clicking on a link in the left navigation or closing their tab. We call this latter action "abandonment"

https://www.mediawiki.org/wiki/Growth/Analytics_updates/Welcome_survey_initial_report

I could actually break that metric down further, if you think that's helpful. Here's the further breakdown of February 2023 Welcome survey data for viwiki:

Response Rate:

53% complete the survey
5% skip (click "Skip this survey")
42% abandon (take some other action that leaves the page or site, such as clicking on a link in the left navigation or closing their tab)

Thank you for the clarification.