We agreed on the following steps to launch the ImageBrowsing A/B test.
Before launch
- Ensure green light from @JScherer-WMF
- Ensure green light from @EBlackorby-WMF and @ovasileva
- Ensure that target wikis are sufficiently internationalized (check translate wiki, solicit help if necessary)
- Arabic ✅
- Chinese needs backport, see T408105#11323773
- French ✅
- Indonesian ✅
- Vietnamese needs backport, see T408105#11323773
- Check patches merged after Mon Oct 27 that need to be backported & backport
Acceptance criteria
- edit FY2025-26 WE3.1 Image Browsing A/B Test's start & end dates. Start on Nov 3, end on Dec 3
- raise traffic to 10% as agreed on Slack
- update the instrumentation spec's sampling rate
- turn on the experiment for Arabic, Vietnamese, French, Chinese, Indonesian wikis only
- data QA
- check that all events are produced on the eqiad.mediawiki.product_metrics.readerexperiments_imagebrowsing and codfw.mediawiki.product_metrics.readerexperiments_imagebrowsing Kafka topics
- T408104: ImageBrowsing: instrumentation production data QA
- T406013: Configure analytics dashboards for ImageBrowsing's A/B test is done
- stop & relaunch new ones as needed, based on T408104's report
Steps to relaunch (AKA rollback plan)
- configure a new experiment on MPIC
- update the experiment name in ReaderExperiments here and here
- update the experiment name in WikimediaEvents here
References
- internal discussion thread
- xLab documentation