While running the A/B test it wasn't 100% clear if the bucketing was working correctly.
We'll add a statsd counter that runs the bucketing code that was used for the A/B test.
$bucket = 'vector.language-test.' . ( $inGroupA ? 'A' : 'B' ); // TODO: Inject this dependency using DI!!1 MediaWikiServices::getInstance() ->getStatsdDataFactory() ->increment( $bucket ) ;
QA
In beta cluster
With the help of a developer, we will need to setup the graph at https://grafana-labs.wikimedia.org/d/000000051/reading-web-beta-dashboard?viewPanel=7&orgId=1
QA should involve checking the 2 buckets are of roughly equal size.
In production
With the help of a dev eloperwe will need to setup a graph on https://grafana.wikimedia.org/d/000000566/overview?orgId=1
QA should involve checking the 2 buckets are of roughly equal size.
QA Results - Beta
| AC | Status | Details |
|---|---|---|
| 1 | ✅ | T286932#7306510 |
QA Results - Prod
| AC | Status | Details |
|---|---|---|
| 1 | ✅ | T286932#7306512 |
