Marking as stalled until we take care of T284666: Add a link: unnecessary articles on units are often suggested and T279434: Add a link: algorithm improvements: Define filter for not linking specific articles types

Where can I add the "inform communities" phase in the checkboxes defined in the task description? :)

In T284481#7148408, @Trizek-WMF wrote:

Where can I add the "inform communities" phase in the checkboxes defined in the task description? :)

Good question. Inform them that it is coming or that we are deploying on a specific date? It would be nice if we didn't have to specify the exact day / time it is happening, but we can do that if you'd like.

@MMiller_WMF per T277356: Add a link: experiment, should this expansion follow the same pattern where we are opting 50% into the structured task for add link and 50% get the old links task, or are we switching everyone (pilot wikis plus this batch) into the structured task for add link?

How is it deployed? If it is a simple configuration change, like at other wikis, I guess we can do it anytime?

It would be nice to have a way for experienced users to try the features. It would be a way to check if we have new cases of links that should be rejected.

In T284481#7151484, @Trizek-WMF wrote:

How is it deployed? If it is a simple configuration change, like at other wikis, I guess we can do it anytime?

It would be nice to have a way for experienced users to try the features. It would be a way to check if we have new cases of links that should be rejected.

It could be enabled like we did for the pilot wikis, where 0% of new accounts were opted into the link-recommendation group, but users could use a browser console snippet to opt-in to the link-recommendation group.

In T284481#7154408, @kostajh wrote:

It could be enabled like we did for the pilot wikis, where 0% of new accounts were opted into the link-recommendation group, but users could use a browser console snippet to opt-in to the link-recommendation group.

Great!

Can we think of an URL parameter instead of a b console snippet?

• Trained the models for each wiki on stat1008 (data should be available in the staging-db)
• Backtesting evaluation does not raise any flags about performance of the model in these wikis; similar to other wikis, i.e. for default threshold 0.5 we get precision ~80% with recall ~40% (or better). I added the detailed results for this round of training/evaluation to the project-page on meta.

After looking at the data in T277355: Add a link: leading indicators, we have decided that this feature is strong enough to go to these additional wikis. The revert rate is low, and user enthusiasm is high. We should continue to proceed with preparing this deployment.

In T284481#7148467, @kostajh wrote:

In T284481#7148408, @Trizek-WMF wrote:

Where can I add the "inform communities" phase in the checkboxes defined in the task description? :)

Good question. Inform them that it is coming or that we are deploying on a specific date? It would be nice if we didn't have to specify the exact day / time it is happening, but we can do that if you'd like.

@MMiller_WMF per T277356: Add a link: experiment, should this expansion follow the same pattern where we are opting 50% into the structured task for add link and 50% get the old links task, or are we switching everyone (pilot wikis plus this batch) into the structured task for add link?

@MMiller_WMF I believe you said that you'd like to roll this out like we did for the pilot wikis: 50% of Growth feature newcomers get link-recommendation structured task, 50% get unstructured links task; all older newcomer users are opted into link-recommendation structured task. Is that correct?

@MMiller_WMF, should we wait for Spanish, since we have T284799: [EPIC] Account creation: LATAM test July 2021 in progress?

@kostajh -- yes, I think we should deploy with the A/B test like we did in our pilot wikis, where half of the users get the structured task and the other half get the unstructured task. @nettrom_WMF, do you agree with that? Would having that additional data from these larger wikis help us see the differences that "add a link" makes?

Also @kostajh -- when is the earliest we would be ready to turn this on? Not because we're in a huge rush, but just so we know what dates we have to choose from?

@Trizek-WMF -- if you think that we should hold off deploying to Spanish because of the Fundraising test, that's fine. Is it because that will be too many changes to communicate to the community at once (I know you're also communicating about mentors)? If you want to hold off, how soon after the Fundraising test would you want to deploy it?

Also,@Trizek-WMF -- for the other wikis on the list (Russian, French, etc.) how much lead time do you want for communication before we deploy?

In T284481#7173617, @MMiller_WMF wrote:

@kostajh -- yes, I think we should deploy with the A/B test like we did in our pilot wikis, where half of the users get the structured task and the other half get the unstructured task. @nettrom_WMF, do you agree with that? Would having that additional data from these larger wikis help us see the differences that "add a link" makes?

@MMiller_WMF : Yes, I agree. The way I read our measurement plan is that we'll extend the A/B test to the additional wikis so they'll have the same 50/50 split between Add a Link and the unstructured task as the four initial wikis currently do. That extension is what enables us to run a shorter experiment to get enough data to assess the difference between the two types of tasks.

Change 701351 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation feature

https://gerrit.wikimedia.org/r/701351

Change 701351 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation feature for more wikis

https://gerrit.wikimedia.org/r/701351

Mentioned in SAL (#wikimedia-operations) [2021-06-24T11:10:59Z] <tgr@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:701351|GrowthExperiments: Enable link recommendation feature for more wikis (T284481)]] (duration: 01m 07s)

In T284481#7173617, @MMiller_WMF wrote:

@Trizek-WMF -- if you think that we should hold off deploying to Spanish because of the Fundraising test, that's fine. Is it because that will be too many changes to communicate to the community at once (I know you're also communicating about mentors)? If you want to hold off, how soon after the Fundraising test would you want to deploy it?

I think we should not deploy two things at the same time. So I would differ the deployment at Spanish Wikipedia to after the Fundraising test.

Also,@Trizek-WMF -- for the other wikis on the list (Russian, French, etc.) how much lead time do you want for communication before we deploy?

I'd prefer to have two full weeks after the announcement, these two weeks would be used to let communities to test the features. So, regarding the A/B testing, we need to have a way to bypass it for experienced users who want to try the feature.

And we also need to T285467: Create an help page for Add a link, distinct from the project page. This page would show how the step by step feature will work, so that users who are not sure about using it, or experienced users, can have an overview of the feature.

@Trizek-WMF we enabled the feature in hidden mode for the wikis listed in this task today. No new users will see the features yet. Current users can use these browser console snippets to opt into the experiment group to see the feature and to opt back out of it again:

ge.utils.setUserVariant('linkrecommendation') // enable link recommendation task on Special:Homepage
ge.utils.setUserVariant('control') // opt out of the experiment

I would wait at least until tomorrow before letting these communities know, because filling up the task pools for these wikis will take some time. The charts at the bottom of https://grafana.wikimedia.org/d/vGq7hbnMz/special-homepage-and-suggested-edits?orgId=1 will provide data when we begin to have some link recommendation tasks available for testing in those wikis.

By the way, @Trizek-WMF & @MMiller_WMF, we have already generated datasets and models for a few other communities during earlier phases of development of Add-Link, and we are in the process of updating those datasets in the production service now. Those wikis are: elwiki, dewiki, simplewiki, and ptwiki. Ptwiki is probably part of the donor project as well, but please let us know if you'd like the link recommendation task to be enabled for any of those other three wikis as part of this second round of deployments.

@Trizek-WMF -- could you please create a separate task for Spanish Wikipedia's deployment so that we don't forget about it?

Also, I think it makes sense to let these wikis have some time in dark mode to test these features, but after this group, I don't think we should have the dark mode period anymore -- I think that would take a lot of communication time before deployment, and not be necessary because we'll have high confidence in the features after deploying to over ten wikis (maybe except for large wikis like English, German, etc.). What do you think?

On Romanian Wikipedia, the homepage says "No suggestions found" for all tasks (Copyediting, Adding links, Updating, Sourcing, Expanding), although yesterday was OK. I think that this problem is related to this deployment.

It seems to be working correctly when I try it.

In T284481#7177222, @FonAfon wrote:

On Romanian Wikipedia, the homepage says "No suggestions found" for all tasks (Copyediting, Adding links, Updating, Sourcing, Expanding), although yesterday was OK. I think that this problem is related to this deployment.

Everything works for me as well. Can you try again perhaps?

In T284481#7177222, @FonAfon wrote:

On Romanian Wikipedia, the homepage says "No suggestions found" for all tasks (Copyediting, Adding links, Updating, Sourcing, Expanding), although yesterday was OK. I think that this problem is related to this deployment.

Filled as T285577: Several wikis have 0 articles for all ORES topics. Sounds to be relevant to you have some topics selected. Also see https://ro.wikipedia.org/wiki/Special:NewcomerTasksInfo.

@Trizek-WMF there are enough tasks now for experienced users to test out the feature using the snippet in T284481#7174973

In T284481#7183377, @kostajh wrote:

@Trizek-WMF there are enough tasks now for experienced users to test out the feature using the snippet in T284481#7174973

Cool! It seems that we are all set to notify communities. :)

Change 703179 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/mediawiki-config@master] GrowthExperiments: Add more wikis to linkrecommendation experiment

https://gerrit.wikimedia.org/r/703179

I tweaked the help page to explain how to enable the tools.

I made a small test at French Wikipedia, and I found a bug: T286215: Encourage the use of PLURAL in MediaWiki:Logentry-growthexperiments-addlink.

I got a few replies to my messages, through various channels. I don't have feedback from all communities, but so far, I only have constructive feedback regarding the quality and accuracy of suggested links.

We should move on. Let's define a date: I'd like to inform the community about what they will see in their recent changes soon after the deployment.

In T284481#7221479, @Trizek-WMF wrote:

I got a few replies to my messages, through various channels. I don't have feedback from all communities, but so far, I only have constructive feedback regarding the quality and accuracy of suggested links.

We should move on. Let's define a date: I'd like to inform the community about what they will see in their recent changes soon after the deployment.

FTR, we decided in slack that this will be deployed tomorrow (2021-07-21), in the EU window.

Change 703179 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Add more wikis to linkrecommendation experiment

https://gerrit.wikimedia.org/r/703179

Mentioned in SAL (#wikimedia-operations) [2021-07-21T11:11:21Z] <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: d6699dae1e96b38b4fae7e8b9817d84b56d2be6c: GrowthExperiments: Add more wikis to linkrecommendation experiment (T284481) (duration: 01m 31s)

The deployment just happened on all the wikis specified in the description. Moving to QA.

The different communities have been notified.

Checked the listed wikis - all seems to be fine. Special:NewcomerTasksInfo shows the sufficient number of tasks for link-recommendation.

MediaWiki:Growthexperiments-addlink-summary-summary/ru exist but hasn't been deployed yet. Otherwise, I think we are good.