Page MenuHomePhabricator

Deploy Growth experiments at Russian Wikipedia
Closed, ResolvedPublic

Description

Note: in this deployment, we should run a welcome survey experiment. This can either be matching the 80/20 split of the rest of the Growth experiment, or it can be an independent 50/50 spit to maximize its speed to significance, whichever is easier.

First

  • Get an agreement from the community to deploy these features: discussion.
  • List people we can contact for the next steps:

Translations

Help panel

  • Help desk: Q11059110 Q5621643 or Q16503
  • Archiving system: Archived after 30 days of inactivity and after 7 days of inactivity for resolved questions
  • Help pages (you can provide either a Wikidata link or a link to your wiki):
  • Search on namespace: 4 "Wikipedia" and 12 "Help" (default)
  • Read more : Q914807

Homepage

Suggested edits

Suggested edits are based on these 5 task-types. At the moment, they can't be substituted. Pleas check maintenance template and links:

TaskDifficulty levelMaintenance templatesLearn more
Copyedit articleEasyQ13859840Q92567905
Add LinksEasyQ5849007Q6521720
Update articleMediumQ5617874Q11128024
Add ReferencesMediumQ5962027 (or Q6192879)Q642335
Expand articleHardQ5866533Q14360362

For templates only, you can suggest more templates that match the task description.

Extra links:

Welcome survey

  • Get an agreement from the community to deploy these features: we have agreed to deploy the welcome survey in an A/B test to check its impact on newcomers.
  • Translate the interface

Understanding first day

Get an agreement from the community to deploy these feature

Anything else?

We use the bot for the template {{hello}}, with automatic substitution of names from this list, and we have FlaggedRevs extension.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentors are really few for the home page :(

Ideally, you would need 26 mentors (~13k new users per month / 500). Less is fine though: when the first mentors will get some questions, they may promote the participation.

Iniquity changed the task status from Stalled to Open.Aug 4 2020, 1:54 PM
Iniquity updated the task description. (Show Details)

Thank you for the update! ;)

You are welcome :) We have a very shaky consensus on the Welcome survey. Is it possible to enable it only for a month and check how it affects on new users?

We already have checked on multiple wikis if it affects new users and it doesn't.

What are the concerns and reasoning around this? This is something we may help with. Let me know!

Mainly arguments: "I would not work with Wikipedia myself if you showed me this poll. Users will think that we want to steal their data"

Iniquity updated the task description. (Show Details)

Mainly arguments: "I would not work with Wikipedia myself if you showed me this poll. Users will think that we want to steal their data"

It is a legit concern. However, I don't see which kind of personal data would be collected there, except favorite topics and reasons why people create an account.

In the survey itself, we highlight the fact that we do not collect any personal data. Maybe one sentence in particular raises concerns?

The Wikimedia Foundation will use your answers to help you start editing [wiki]
Almost every page can be edited by anyone, and hundreds of changes are made every hour to add content, correct mistakes, and improve readability. The following pages can help if you’re interested in contributing.

New users can also skip this survey.

Which proportion of users raised this up?

Which proportion of users raised this up?

Enough to cause concern :) You can google translate this part of consensus. I just need data about our test to convince them.

Like the other tools, the welcome survey is deployed to 80% of newcomers. We keep a tracking group of 20% to see if the survey impacts account creations. It is done for every wikis, except small ones (and you're not a small one).

It is equivalent of the one-month trial you suggest. It is even better, because longer: one bad month can happen (less registrations in August because of summer vacations, for instance), but three bad months in a row are less likely to happen.

Would this reassure your community?

Thank you for your help! ;)

I think yes, but in any case I will need the same report :)

@Iniquity -- yes, we can specially run the experiment around the welcome survey to check whether it depresses the number of newcomers who make edits, and we will be able to report back after about 6 weeks.

MMiller_WMF updated the task description. (Show Details)
MMiller_WMF added a subscriber: Catrope.

@Iniquity -- yes, we can specially run the experiment around the welcome survey to check whether it depresses the number of newcomers who make edits, and we will be able to report back after about 6 weeks.

It would be cool :)

Hi :) When it is appear? :)

Hi! The team will discuss about it today.

In the meantime, please check on the task description. I found that 3 links are missing.

On the welcome survey, we have recently added a new question about languages people understand, so that we can know if suggesting translations as a task would be a good idea. This needs some extra work on translations (2 more messages).

In the meantime, please check on the task description. I found that 3 links are missing.

Oh, I see :) Thanks, we try to fix it soon as possible.

@Trizek-WMF I suddenly thought, why is it required to specify a template, and not a category for 'Suggested edits'? We have just half of the articles using a universal template {{rq}} with additional parameters.

We use templates because they are relied to one category only (mostly). This is not the case with categories, that can be attached to multiple templates (like the "multiple issues" template). These templates are confusing to users, since they can't find a clear direction when they open the article.

Polish Wikipedia had the same issue as you have with templates and we have a fix. It requires some work on your side.

We suggest that you create "dummy" templates for each task types we need and then and then add {{#ifexist:maintenance-{{{1}}}|{{maintenance-{{{1}}}}}}} to the main template to have it working. It is very unlikely that people would use the dummies, since finding templates relies on search, and search suggests the most used templates.

Let me know if you have questions!

That is, it will search the entire template tree, hmm... It's okay that our template may have several issues in one? I mean, it looks a lot like "multiple issues" from enwiki.

We try not to use the "multiple issue template. :)

Have one template that lists numerous things confuses people: "what's important?" "If I have to do X but I read that X, Y and Z must be completed, what should I do?"

We try to select articles that have one template, which clearly tells the user what to do. It is easy to fix one thing instead of multiple things at once. Fixing multiple things at one is for experienced users.

We try not to use the "multiple issue template. :)

Have one template that lists numerous things confuses people: "what's important?" "If I have to do X but I read that X, Y and Z must be completed, what should I do?"

We try to select articles that have one template, which clearly tells the user what to do. It is easy to fix one thing instead of multiple things at once. Fixing multiple things at one is for experienced users.

Then we have a problem :) The simple copyedit template has only 50 inclusions. We can change it to 'wikify' template.

Then we have a problem :) The simple copyedit template has only 50 inclusions. We can change it to 'wikify' template.

Please do. :) You can edit the task description to either change wikidata link or replace it with a direct link to the template on ru.wp. Do other templates have enough tasks?

Do other templates have enough tasks?

Everything seems to be fine with the rest of the templates. I will think about how best to do it.

@Catrope is going to do the setup this week, which will identify whether there are any other issues that need to be addressed by the community before we deploy.

@Catrope is going to do the setup this week, which will identify whether there are any other issues that need to be addressed by the community before we deploy.

I am glad to hear it :)

Like the other tools, the welcome survey is deployed to 80% of newcomers. We keep a tracking group of 20% to see if the survey impacts account creations. It is done for every wikis, except small ones (and you're not a small one).

This is actually no longer true: we now run the welcome survey at 100% on all wikis, large and small, new and old. But from the discussion here (and the fact that T259768 exists), it's clear that an A/B test is desired, so I'll configure the welcome survey to run at 80% on Russian Wikipedia.

@MMiller_WMF @nettrom_WMF Let me know if you want a different ratio than 80/20, it's easy to change as long as it's multiples of 10%.

@Iniquity Could you help me out with the help panel links? You've provided the targets for these links (the pages the link takes you to when you click it), but not the label of each link (the text that is displayed to the user). For example, "Introduction to images with VisualEditor" might be the name of a page, but for newcomers we probably want to use "How to insert an image" as the text for that link. I've made some guesses, but I don't speak Russian, so most of my guesses are probably wrong. Could you please tell me what corrections I need to make to the list below?

@Trizek-WMF: Since this is a common thing I have to ask for, could you perhaps change the template you use for these tasks to make it clearer that people need to provide both the targets and the labels for these links? Also, Wikidata QIDs are fine, I can click through and find the page they refer to, but it's slightly more convenient if people provide links to the pages themselves.

Change 621108 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] Enable GrowthExperiments on ruwiki

https://gerrit.wikimedia.org/r/621108

Task types page ready on the wiki: https://ru.wikipedia.org/wiki/MediaWiki:NewcomerTasks.json

This will be ready to go as soon as we hear back from @Iniquity about the link labels (see above). Once we have the corrections for those, we can schedule this deployment for whenever we want (but preferably a Monday or a Wednesday at 18:00 UTC).

@Iniquity Could you help me out with the help panel links? You've provided the targets for these links (the pages the link takes you to when you click it), but not the label of each link (the text that is displayed to the user). For example, "Introduction to images with VisualEditor" might be the name of a page, but for newcomers we probably want to use "How to insert an image" as the text for that link. I've made some guesses, but I don't speak Russian, so most of my guesses are probably wrong. Could you please tell me what corrections I need to make to the list below?

Oh, I didn't think about it :)

Oh, I didn't think about it :)

That's OK! You're not the first person to miss this, which means that it's not your fault and that the instructions aren't clear enough :)

Thanks! I've updated the patch with these link labels.

This deployment is now ready to be scheduled. When @Trizek-WMF is back from vacation next week, I'll ask him to give this the final go-ahead and set a deployment date.

This comment was removed by Trizek-WMF.

@Iniquity, is there any event scheduled within your community that would influence this date? If not, we can pick any time this week, like on Wednesday, August 26. @Catrope, do you have any constraints?

I also note this on the task description:

We use the bot for the template {{hello}}, with automatic substitution of names from [a] list

We plan to work on a way to align these two lists if it is proven that the welcome message is useful. A research project has been led on French Wikipedia, testing the welcome message. We hope to have the results soon.
In the meantime, French Wikipedia has two lists, one for the welcome message and the other one for the Growth Homepage. A bot checks for differences and edit the two pages accordingly.

@Iniquity, is there any event scheduled within your community that would influence this date? If not, we can pick any time this week, like on Wednesday, August 26.

It seems to me that you can set up it any day :)

We plan to work on a way to align these two lists if it is proven that the welcome message is useful. A research project has been led on French Wikipedia, testing the welcome message. We hope to have the results soon.
In the meantime, French Wikipedia has two lists, one for the welcome message and the other one for the Growth Homepage. A bot checks for differences and edit the two pages accordingly.

I'm also thinking about merging these lists. I would gladly ditch the 'hello' template altogether.

I had to coordinate with @Catrope, and, due to timezones, we missed our replies. @Iniquity, the deployment will start tomorrow, August 27. Catrope will provide the timeframe.

I had to coordinate with @Catrope, and, due to timezones, we missed our replies. @Iniquity, the deployment will start tomorrow, August 27. Catrope will provide the timeframe.

Ok, I am waiting for you :)

I will deploy this tomorrow (Thursday August 27) between 18:00-19:00 UTC (21:00-22:00 Moscow time)

Catrope set Due Date to Aug 27 2020, 6:00 PM.Aug 26 2020, 5:09 PM

Change 621108 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable GrowthExperiments on ruwiki

https://gerrit.wikimedia.org/r/621108

Change 622861 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] Actually enable homepage on ruwiki

https://gerrit.wikimedia.org/r/622861

Change 622861 merged by jenkins-bot:
[operations/mediawiki-config@master] Actually enable homepage on ruwiki

https://gerrit.wikimedia.org/r/622861

Mentioned in SAL (#wikimedia-operations) [2020-08-27T18:31:46Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable GrowthExperiments on ruwiki (T257490) (duration: 01m 03s)

This is now deployed. Seems to be working at first glance.

Wow! Thanks :) I am going to check :)

Etonkovidova subscribed.

Checked (also on mobile) no issues have been found - moving to PM Review.

Oh, I forgot to specify that the topics are added to the top on our help desk, it is possible to make the same behavior for 'help panel' posts?

Oh, I forgot to specify that the topics are added to the top on our help desk, it is possible to make the same behavior for 'help panel' posts?

We don't support that right now. I don't think it will be much effort to add support for it, but we would need some time to do that. It's not just a switch we can flip (yet). I've filed T261714 for this.

Oh, I forgot to specify that the topics are added to the top on our help desk, it is possible to make the same behavior for 'help panel' posts?

We don't support that right now. I don't think it will be much effort to add support for it, but we would need some time to do that. It's not just a switch we can flip (yet). I've filed T261714 for this.

Thank! I think we're not the only wiki who might need it :)

@Iniquity - the most recent question - Википедия:Форум/Помощь_начинающим&diff=next&oldid=109426145 seems to be a test question and was made on Sep 23 and deleted (the tag #отменено).
You may check - ask a test question via Help panel and the delete the test question. I checked in our test env; posting questions via Help panel seems to to be working.

@Iniquity - the most recent question - Википедия:Форум/Помощь_начинающим&diff=next&oldid=109426145 seems to be a test question and was made on Sep 23 and deleted (the tag #отменено).
You may check - ask a test question via Help panel and the delete the test question. I checked in our test env; posting questions via Help panel seems to to be working.

Yes, we tested that. But 2 weeks no questions from newscomers, and it is strange.

I checked on Википедия:Форум/Помощь_начинающим and I saw no new messages since September 23. Is there anything on the translation that would make things unclear, and prevent users from using the Help panel? Have you posted test messages?

Have no questions may happen as well. :)

On the other hand, I notice people who make suggested edits and contact their mentors. This part works! :)

I checked on Википедия:Форум/Помощь_начинающим and I saw no new messages since September 23. Is there anything on the translation that would make things unclear, and prevent users from using the Help panel? Have you posted test messages?

Have no questions may happen as well. :)

On the other hand, I notice people who make suggested edits and contact their mentors. This part works! :)

The translation, it seems, has not changed. Is it possible that the button just stopped showing to beginners?

I tried to post a message on help desk using the Help panel, and it worked.

It also perfectly works for asking your mentor.

The most reasonable explanation is that people aren't asking questions. It sometimes happens. :)

We just noticed that russian aliases for special pages arent working:

You're welcomed. Thank you for reporting it!

By the way:

Oh, I forgot to specify that the topics are added to the top on our help desk, it is possible to make the same behavior for 'help panel' posts?

New messages will soon be posted at the top by default on your wiki. Please see T268227: Apply posting new help desk questions at the top of the page to the wikis needing them for the details!

New messages will soon be posted at the top by default on your wiki. Please see T268227: Apply posting new help desk questions at the top of the page to the wikis needing them for the details!

Wow:) This is great news! Thanks!

New messages will soon be posted at the top by default on your wiki. Please see T268227: Apply posting new help desk questions at the top of the page to the wikis needing them for the details!

Wow:) This is great news! Thanks!

It's been resolved. :)

New messages will soon be posted at the top by default on your wiki. Please see T268227: Apply posting new help desk questions at the top of the page to the wikis needing them for the details!

Wow:) This is great news! Thanks!

It's been resolved. :)

Awesome news, thanks! :)