As part of T377631: Add a link (Structured task): Release to a subset of newcomers on English Wikipedia, we want to deploy Add Link to a percentage of English Wikipedia newcomers. To be able to do that, we introduced the no-link-recommendation variant in T377787. This task tracks verifying Add Link is ready for partial deployment to English Wikipedia (see T377631 for the deployment checklist).
Verification options
There are multiple different ways how to verify Add Link's readiness for deployment. We have agreed to pursue verification option #3 (Deploy Add Link partially to testwiki).
- Switch test accounts to no-link-recommendation variant manually: Starting Nov 19, 05:00 UTC (with wmf.4 train), the no-link-recommendation variant should be testable at test.wikipedia.org. We can manually switch to that variant using ge.utils.setUserVariant( 'no-link-recommendation' ) and verify it works correctly. This variant is already available in beta (and we can easily backport if we want to speed testing up).
- Deploy Add Link partially to enwiki beta: We can deploy the full experiment to English Wikipedia beta, as it would be later deployed to English Wikipedia production. Assuming Beta's stability lets us, we should be able to do an end-to-end test. This might give us more confidence everything is working correctly, including the variant assignment.
- Deploy Add Link partially to testwiki: We can deploy the full experiment to test.wikipedia.org, as it would be later deployed to English Wikipedia. This would allow us to do an end to end test in production, eliminating "Beta being Beta" as a possible way to miss bugs.
- NOTE: Because T288022: Support multidimensional user variants in GrowthExperiments is unresolved, we would have to undeploy the Community updates experiment from test.wikipedia.
- Deploy Add Link partially to test2wiki: We can deploy the full experiment to test2.wikipedia.org, as it would be later deployed to English Wikipedia. This would allow us to do an end to end test in production, eliminating "Beta being Beta" as a possible way to miss bugs. It would also ensure Community Updates are still enabled at test.wikipedia.org.
- NOTE: To do this, we would need to set Add Link up at test2.wikipedia.org (currently, it does not work there properly). While that is possible, it would mean more work we would need to do.
- ALTERNATIVE: If we want to keep both Community Updates and partial Add Link, we can do option 3, and also enable Community Updates at test2.wikipedia.org (that would be much simpler to do than setting up Add Link at another wiki).
Checklist
- @KStoller-WMF to decide how to verify Add Link's readiness for partial deployment
- T380204: Deploy Add Link to a proportion of test.wikipedia.org users
- @Etonkovidova to confirm partial deployment of Add Link works as intended.
QA testing notes:
@KStoller-WMF is hoping we can test:
Given I'm an Admin viewing Community Configuration of Suggested Edits.
When the Add a link (Structured task) in enabled on my wiki, but only to a percentage of newcomers due to an experiment,
Then I still have the ability to disable the task in Community Configuration