Page MenuHomePhabricator
Create Task
Maniphest T140292

A/B Test TextCat settings on non-WP projects
Open, MediumPublic
Actions

Description

We want to spread the usefulness of Language Identification (via TextCat) to non-Wikipedia wikis.

Rather than do a time-consuming manual analysis for each wiki project, we could do an A/B test on some/all projects in the same language using the default configs for the Wikipedia project in that language (for which analysis is done).

Such A/B tests would give us insight into whether the TextCat configs can be straightforwardly shared across projects in the same language. If so, it would help us be able to apply language detection to more of the long tail of wiki projects.

Related Objects
Search...

Event Timeline

TJones created this task.Jul 13 2016, 7:50 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 13 2016, 7:50 PM
TJones added a parent task: T118278: [EPIC] Improve Language Identification for use in Cirrus Search.Jul 13 2016, 7:59 PM
EBernhardson added a subscriber: EBernhardson.Jul 14 2016, 10:17 PM
CKoerner_WMF added a subscriber: CKoerner_WMF.Jul 14 2016, 10:17 PM
debt triaged this task as Medium priority.EditedJul 14 2016, 10:17 PM
debt moved this task from needs triage to This Quarter on the Discovery-Search board.
debt added a subscriber: debt.

Let's pick one or two other project wiki's in the same language and determine if we have enough data to do a test on already or if we have to do more data gathering. Maybe Wiktionary and Wikispecies (in EN) would be good candidates to start off with.

debt added a subscriber: mpopov.Aug 5 2016, 7:39 PM

notes based on a conversation with @TJones:

This seems like an easy A/B test to set up and run - pick a few languages, pick a few non-WP projects, configure, run for a couple of weeks and then get some analysis on it.

debt created subtask T142584: Investigate creating mocks for UI component of this A/B test.Aug 10 2016, 4:12 PM
debt closed subtask T142584: Investigate creating mocks for UI component of this A/B test as Declined.Nov 2 2016, 1:28 PM
TJones moved this task from This Quarter to Tech Debt/Misc on the Discovery-Search board.Oct 24 2017, 5:35 PM
debt moved this task from Tech Debt/Misc to Language Stuff on the Discovery-Search board.Jan 29 2019, 6:40 PM
CKoerner_WMF removed a subscriber: CKoerner_WMF.Dec 12 2019, 4:15 PM
Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL