Page MenuHomePhabricator

Run wikidata entitiy autocomplete AB test in de, fr, es
Closed, ResolvedPublic

Description

With the new profiles code deploying this week (T213106) we have everything in place to run the AB test for the new profiles (T213105). This should be run in the same way as the previous english test (T211033), and analyzed with the same report/metrics.

Details

Related Gerrit Patches:
operations/mediawiki-config : masterPromote new wbsearchentities profiles to default
operations/mediawiki-config : masterTurn off wbsearchentities ab test in de, fr, es
operations/mediawiki-config : masterTurn on wbsearchentities ab test in de, fr, es
operations/mediawiki-config : masterAdd wbsearchentities profiles for de, fr, es

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 23 2019, 8:03 PM
EBernhardson renamed this task from Run wikidata entitit autocomplete AB test in de, fr, es to Run wikidata entitiy autocomplete AB test in de, fr, es.Jan 23 2019, 8:07 PM

Change 484334 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/mediawiki-config@master] Add wbsearchentities profiles for de, fr, es

https://gerrit.wikimedia.org/r/484334

Change 486154 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/mediawiki-config@master] Turn on wbsearchentities ab test in de, fr, es

https://gerrit.wikimedia.org/r/486154

EBernhardson updated the task description. (Show Details)Jan 23 2019, 8:56 PM

Change 484334 merged by jenkins-bot:
[operations/mediawiki-config@master] Add wbsearchentities profiles for de, fr, es

https://gerrit.wikimedia.org/r/484334

Mentioned in SAL (#wikimedia-operations) [2019-01-25T00:37:57Z] <ebernhardson@deploy1001> Synchronized wmf-config/WikibaseSearchSettings.php: SWAT T214515 gerrit:484334: Add wbsearchentities profiles for de, fr, es (duration: 00m 45s)

Change 486154 merged by jenkins-bot:
[operations/mediawiki-config@master] Turn on wbsearchentities ab test in de, fr, es

https://gerrit.wikimedia.org/r/486154

Mentioned in SAL (#wikimedia-operations) [2019-01-25T00:46:27Z] <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT T214515 gerrit:486154: Turn on wbsearchentities ab test in de, fr, es (duration: 00m 46s)

EBernhardson updated the task description. (Show Details)

Change 488588 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/mediawiki-config@master] Turn off wbsearchentities ab test in de, fr, es

https://gerrit.wikimedia.org/r/488588

Change 488588 merged by jenkins-bot:
[operations/mediawiki-config@master] Turn off wbsearchentities ab test in de, fr, es

https://gerrit.wikimedia.org/r/488588

Mentioned in SAL (#wikimedia-operations) [2019-02-08T00:22:02Z] <ebernhardson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT gerrit:488588 phab:T214515 Turn off wikidata wbsearchentities ab test in de, fr, es (duration: 02m 55s)

AB test reports published. Clicks@1 improved in all languages with no regressions in other metrics. Clicks@1 starts around 78-80% and improves by 2-5% to 83-85% depending on language. Will deploy these profiles as the new defaults.



EBernhardson updated the task description. (Show Details)Feb 11 2019, 7:31 PM

Change 489780 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/mediawiki-config@master] Promote new wbsearchentities profiles to default

https://gerrit.wikimedia.org/r/489780

Change 489780 merged by jenkins-bot:
[operations/mediawiki-config@master] Promote new wbsearchentities profiles to default

https://gerrit.wikimedia.org/r/489780

Mentioned in SAL (#wikimedia-operations) [2019-02-12T00:28:05Z] <ebernhardson@deploy1001> Synchronized wmf-config/WikibaseSearchSettings.php: gerrit:489780 T214515 Promote new wbsearchentities profiles to default in de, fr, es (duration: 00m 46s)

debt closed this task as Resolved.Feb 15 2019, 7:02 PM