Page MenuHomePhabricator

Personalized first day: potential survey wording changes
Closed, DeclinedPublic


Since deploying the welcome survey and seeing initial data coming in, we have started talking about how the questions and responses could be rephrased so that responses match our intent.

We want to be deliberate about when we make any changes so that our data analysis doesn't get mixed up. We should not make any changes before Variation A has run for a month.

@RHo has been collecting some of these thoughts.

  • In the question, "Have you ever edited Wikipedia?", the response "No, I didn't know I could edit Wikipedia" is getting a higher response rate than expected. This might be because it includes both people who mean, "No, I didn't know I could edit Wikipedia until I read this question" and people who mean, "No, I didn't know I could edit Wikipedia until earlier today, which is why I am currently creating my account." One idea to help here might be to simply exclude the word "No", so that it says, "I didn't know I could edit Wikipedia." And the response that currently reads, "No, for other reasons" would read, "No, I have never edited".
  • We may eventually want communities to customize the options in the question about why users created their account so that we can align their experience with their objective. For instance, if Czech Wikipedia were to make a big push on Wiki Loves Monuments, we may want them to add that as a welcome survey option for why users are creating their account, and then there could be custom homepage content for that program.

Event Timeline

MMiller_WMF moved this task from Needs Discussion/Analysis to Upcoming Work on the Growth-Team board.

We decided that first we want to crosstab "Have you ever edited" with "Why did you create your account" to see if we see a lot of people who said they are creating an account to edit, but also say they didn't know they could edit.

After @nettrom_WMF does that, we can decide if/when to change the survey wording.

I just removed the following idea from the description because it is going to be worked on in T232400:

  • As we get close to using the welcome survey to personalize the newcomer homepage, we are thinking about the most important things to know about newcomers. These are some responses we may want to add to the first question about why users are creating their account:
    • I am at an editing event. This response could maybe open up more options to say the specific event or kind of event.
    • I am completing a school assignment.
    • I want to add a photo.

On T216081: Welcome survey: analyze free-text responses, @JTannerWMF identified these topics as good to add to the list:

Additionally, in regards to topic selection, for Korean Wiki, we should add an option for animals, anime, brands, people, and languages. For Czech Wiki we should add animals, brands, people, travel, nature, structures/architecture.

nettrom_WMF moved this task from Triage to Backlog on the Product-Analytics board.

@MMiller_WMF : I'm un-assigning myself from this task as I don't see this as work that I'd be doing. Secondly, I'm not sure if this is a task that needs to be kept open, as we've used other tasks for specific survey changes (e.g. dropping topics, adding languages).

kzimmerman added subscribers: KStoller-WMF, kzimmerman.

Per Morten's comment above, I'm removing Product Analytics.

@KStoller-WMF - can this task be declined?

@kzimmerman Yes, I think it makes sense to decline this as we don't have any immediate plans to work on the Welcome survey, and this task is rather outdated.