Page MenuHomePhabricator
Create Task
Maniphest T137163

Part Deux: TextCat A/B test for Language Identification - specification
Closed, ResolvedPublic
Actions

Description

This is a continuation - with modifications - of a test that started in https://phabricator.wikimedia.org/T121543.

We needed to modify the event logging in order to get reliable data to analyze.

Here's a list of issues with EL/data from the first run: T137158. In summary: 1. we didn't record if the clickthrough was on an interwiki result; 2. something funky is going on with the third value of the extra parameters field; 3. that third value needs to be explicitly 0 when the number of results returned is 0, rather than just missing.

Details

SubjectRepoBranchLines +/-
Textcat search satisfaction subtest for multiple wikismediawiki/extensions/WikimediaEventsmaster+48 -7
Textcat search satisfaction subtest for multiple wikismediawiki/extensions/WikimediaEventswmf/1.28.0-wmf.6+48 -7
Customize query in gerrit

Related Objects
Search...

Event Timeline

debt created this task.Jun 6 2016, 10:56 PM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptJun 6 2016, 10:56 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
debt added a parent task: T121543: Do an A/B Tests on Other Wikis with TextCat for Language Identification.Jun 6 2016, 10:56 PM
mpopov updated the task description. (Show Details)Jun 6 2016, 11:02 PM
debt added a parent task: T137167: Part Deux: TextCat A/B test for Language Identification - create and deploy.Jun 6 2016, 11:12 PM
debt added a parent task: T137170: Part Deux: TextCat A/B test for Language Identification - analysis of results.Jun 6 2016, 11:20 PM
debt added a parent task: T137169: Part Deux: TextCat A/B test for Language Identification - turn off test.
debt added a parent task: T137168: Part Deux: TextCat A/B test for Language Identification - ensure test is going well.
EBernhardson added a comment.Jun 16 2016, 4:48 PM

To make deployment a bit easier fixing the test and re-deploying it are being done in the same patch (linked to both this ticket and the ticket for turning it on).

EBernhardson claimed this task.Jun 16 2016, 4:49 PM
EBernhardson edited projects, added Discovery-Search (Current work); removed Discovery-Search.
EBernhardson moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.
debt triaged this task as Medium priority.Jun 16 2016, 5:24 PM
gerritbot subscribed.Jun 16 2016, 6:03 PM

Change 293432 had a related patch set uploaded (by EBernhardson):
Textcat search satisfaction subtest for multiple wikis

https://gerrit.wikimedia.org/r/293432

gerritbot added a project: Patch-For-Review.Jun 16 2016, 6:04 PM
gerritbot added a comment.Jun 16 2016, 8:02 PM

Change 293432 merged by jenkins-bot:
Textcat search satisfaction subtest for multiple wikis

https://gerrit.wikimedia.org/r/293432

gerritbot added a comment.Jun 16 2016, 8:07 PM

Change 294773 had a related patch set uploaded (by EBernhardson):
Textcat search satisfaction subtest for multiple wikis

https://gerrit.wikimedia.org/r/294773

gerritbot added a comment.Jun 16 2016, 11:09 PM

Change 294773 merged by jenkins-bot:
Textcat search satisfaction subtest for multiple wikis

https://gerrit.wikimedia.org/r/294773

EBernhardson moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Jun 20 2016, 8:42 PM
• Deskana closed this task as Resolved.Jul 7 2016, 7:20 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL