Hi @KStoller-WMF! @Sgs and me discussed this task briefly this morning. I'd like to share our observations and add technical considerations to some of the questions asked in the description.

Firstly, I'd like to share some context that would be useful for discussion about the questions. The task essentially asks for an A/B experiment over the link-recommendation task type, which would be carried only on the English Wikipedia. Unfortunately, we already removed the relevant A/B testing code (T294737#7504870), which means we would need to reintroduce it before being able to do this task. To do that, we would need to introduce a new variant called no-link-recommendation, which would disable link-recommendation for users who are in it. In other words, we would have those variants available:

• control (the default one): link-recommendation enabled, assuming it is also enabled in Community Configuration,
• no-link-recommendation: link-recommendation is not available, regardless of the Community Configuration setting

This is a change to how the Add Link experiment originally worked (link-recommendation disabled in control and enabled only for users who are in the link-recommendation variant). This is because the definitions of all the variants needs to be the same across all of our wikis, and we do not want to disable Add Link at any wiki where it is already deployed to. In other words, this would be a reverse experiment, similar to what we did with the oldimpact variant (which was disabling New Impact).

With the above said, diving into the questions from the descriptions:

Should we release to a percentage of newly created accounts or to a percentage of all accounts (that use the Homepage)?

Since we would be using the (to-be-introduced) no-link-recommendation variant, we would actually control the percentage of users who do NOT have access to Add Link. I can see several variants of how this could work like:

1. Assign the no-link-recommendation variant to N% of newly registered users
2. Assign the no-link-recommendation variant to N% of all users
3. Assign the no-link-recommendation variant to all legacy accounts. Also assign it to N% of newly registered users.

Implementing options 2 and 3 requires T376266 (and T377713 would be also helpful, although not a strict blocker). Fortunately, T376266 is already in QA. We cannot separately control the percentage of legacy and all users, but we can assign a certain variant to all legacy users (irrespective of the set percentage).

Option 1 would give Add Link access to all legacy users and a selected percentage of new users. I think this would not be acceptable for the English Wikipedia community, as they have a great number of legacy users. Even if a small percentage of them still use Special:Homepage, option 1 has a great risk of overflowing the English Wikipedia community with too many Add Link edits. For the record, English Wikipedia has about 3 millions of legacy users.

Option 2 would give Add Link access to a selected percentage of all Homepage users (legacy + new users combined). This is definitely much safer than enabling Add Link to all legacy users, but given the very high number of legacy users we're working with, it still feels a little bit unsafe.

Option 3 means releasing only to newly registered users. This is a variable we're most used to be working with, and one where we can estimate the impact best. Hence, I think we should pick this one, at least for now. We can follow-up with scaling to legacy users once English Wikipedia verifies the feature works well enough.

FWIW, we were hit with accidentally deploying to too many users before. When we were actively working on scaling mentorship to Spanish Wikipedia, we first released to 25% of new accounts (and all legacy users), the mentors reported getting overworked and as such, we decreased the percentage of new users to 15% (see T285235 for details). Since this has already hit us in the past, I think we shouldn't risk doing that again (this time knowingly).

Technically, there aren't any concerns with releasing to legacy users in the first stage. If the English Wikipedia is fine with including them (them

What percentage should we release to?

No technical constraints, any percentage works.

Socially: I'd go with 10% of new accounts. It is relatively close to the 2% trial we originally had with Growth features at enwiki, and it is also not terribly low. For similar rollouts, we went with 15% for eswiki mentorship and 2% for first Growth rollout (next step was 25%).

What are the default tasks if I don't have access to the "Add a link" structured tasks?

The same as English Wikipedia has available now: Copyedit, template-based Add Link, Find references, Update articles and Expand short articles. The template-based Add Link would be heavily impacted by the limited size of the task pool, so it might not be visible to most newcomers (that said, this is the status quo).

Will turning off the task via Community Configuration still work if Growth has released the task to a small percentage (which isn't represented in Community Configuration)?

It should. Add Link would be only available if it is both enabled in Community Configuration and the user is not in the special no-link-recommendation variant. Community Configuration wouldn't make it obvious the feature is only available to a certain percentage of users, but given it is a temporary measure only, I believe it is acceptable for now. Unchecking the box should work no matter what though.

Hope this clarifies!

Excellent, this is extremely helpful!

Unfortunately, we already removed the relevant A/B testing code

Ouch. I guess I didn't understand the full implications of removing that code.

Option 3 means releasing only to newly registered users.

I agree we should go with this option. I think it's more ideal from a patroller perspective and in alignment with the WE1.2 Key Result (to increase the percentage of new account holders who edit constructively).

No technical constraints, any percentage works.

OK, thanks. I'll check in on the enwiki discussion and then update this task.

Add Link would be only available if it is both enabled in Community Configuration and the user is not in the special no-link-recommendation variant. Community Configuration wouldn't make it obvious the feature is only available to a certain percentage of users, but given it is a temporary measure only, I believe it is acceptable for now.

Agreed. I just wanted to make sure admins still have a disable option. I don't want to accidentally violate community trust by releasing the feature with a Community Configurable "off switch" that doesn't actually work.

In T377631#10248437, @KStoller-WMF wrote:

Excellent, this is extremely helpful!

Glad to hear that! One thing I forgot to mention originally: fulfilling this task means we would be unable to deploy Community Updates to English Wikipedia (until Add Link is at 100% there). This is because we can run at most one experiment per wiki (until we implement T288022), and partial Add Link deployment counts as an experiment.

Unfortunately, we already removed the relevant A/B testing code

Ouch. I guess I didn't understand the full implications of removing that code.

The removal happened almost 3 years ago, it's not a recent decision :). I filled T377787: Add Link (structured): Introduce the no-link-recommendation variant to create the variant we'll need here, as that is independent on actually deploying Add Link to enwiki.

Add Link would be only available if it is both enabled in Community Configuration and the user is not in the special no-link-recommendation variant. Community Configuration wouldn't make it obvious the feature is only available to a certain percentage of users, but given it is a temporary measure only, I believe it is acceptable for now.

Agreed. I just wanted to make sure admins still have a disable option. I don't want to accidentally violate community trust by releasing the feature with a Community Configurable "off switch" that doesn't actually work.

Makes sense. This is probably something we would want to double check before going ahead with a production deployment.

One thing I forgot to mention originally: fulfilling this task means we would be unable to deploy Community Updates to English Wikipedia (until Add Link is at 100% there). This is because we can run at most one experiment per wiki (until we implement T288022), and partial Add Link deployment counts as an experiment.

Noted. I mentioned this in the enwiki discussion as I want to ensure we communicate that we can't indefinitely stay rolled out to just a percentage of newcomer accounts.

The removal happened almost 3 years ago, it's not a recent decision :)

Ohhhh gotcha, I mistakenly thought this might relate to some of the other clean up work we have in progress, like T350077: Drop support for the old Impact module.

This is probably something we would want to double check before going ahead with a production deployment.

Agreed. I added a QA testing note to the task description so we don't forget.

Blocked by T379681: community-updates-module variant is assigned outside of Growth pilot wikis.

Blocked by T380161: Verify Add Link's readiness for partial deployment.

Given that we have completed: T380161: Verify Add Link's readiness for partial deployment, and we have Community support at enwiki for a 2% release, we should now finalize a release date.

I'm asking that we get a final Movement Communications OK from @Trizek-WMF before release, but once we have that OK and @Trizek-WMF or another Growth team member follows up with a release date on the associated enwiki thread, then we are good to release.

We have agreed to a Monday, November 25th 2024 release, and I have updated the enwiki community here.

Change #1095126 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[operations/mediawiki-config@master] [Growth] enwiki: Deploy Add Link to 2% of new users

https://gerrit.wikimedia.org/r/1095126

Change #1095126 merged by jenkins-bot:

[operations/mediawiki-config@master] [Growth] enwiki: Deploy Add Link to 2% of new users

https://gerrit.wikimedia.org/r/1095126

Mentioned in SAL (#wikimedia-operations) [2024-11-25T19:24:24Z] <urbanecm@deploy2002> Started scap sync-world: Backport for [[gerrit:1095126|[Growth] enwiki: Deploy Add Link to 2% of new users (T377631)]]

Mentioned in SAL (#wikimedia-operations) [2024-11-25T19:28:37Z] <urbanecm@deploy2002> urbanecm: Backport for [[gerrit:1095126|[Growth] enwiki: Deploy Add Link to 2% of new users (T377631)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-11-25T19:36:24Z] <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1095126|[Growth] enwiki: Deploy Add Link to 2% of new users (T377631)]] (duration: 11m 59s)

Deployed. I also enabled Add Link via Community Configuration, to ensure it is actually visible to users (the 2% we enabled for). Moving to QA!

Checked on enwiki wmf.4 - RecentChanges show edits tagged with Suggested:add links made by non-new accounts.

Mentioned in SAL (#wikimedia-operations) [2024-11-26T16:40:51Z] <urbanecm> mwscript-k8s -f userOptions.php -- --wiki=enwiki --old=control --delete 'growthexperiments-homepage-variant' # T379146, T377631

In T377631#10356043, @Etonkovidova wrote:

Checked on enwiki wmf.4 - RecentChanges show edits tagged with Suggested:add links made by non-new accounts.

Thanks for the report! That wasn't supposed to happen. The problem was T379146. I deleted the accidental control rows from user_properties, and now only new users should have Add Link access. Let me know if you spot a similar issue, please!

Thx, @Urbanecm_WMF - I'll monitor more for couple of days the RC filter with newcomers, learners, and experienced users. Last timestamp for a Suggested: add link edit made by non-newcomer.