|mediawiki/extensions/WikimediaEvents : master||Textcat search satisfaction subtest for multiple wikis|
|mediawiki/extensions/WikimediaEvents : wmf/1.28.0-wmf.6||Textcat search satisfaction subtest for multiple wikis|
|mediawiki/extensions/CirrusSearch : master||Add implementation for TextCat language detection|
- Mentioned Here
- T132466: Lang ID Eval Sets for Italian, German, Spanish, and French
T134431: Re-Optimize Italian, German, Spanish, and French TextCat Languages by Recall
T136034: [EPIC] Estimate the "wasted" computational cost of recall- vs precision-focused configs
T121541: Create Properly Weighted Language Identification Evaluation Sets for Top N Other Wikis
T121542: Write and deploy an A/B Test on enwiki using TextCat for Language Identification
@TJones I was looking at these tasks, and wondering if the blockers here are really blockers for running an extra A/B test. @EBernhardson and I think they may not be, and that based on your work so far that we could run a test right now, but we don't know this stuff as well as you do, so we'd like to ask you. Thoughts?
They are and they aren't—what a helpful answer!
The tasks are really too general, and at the earliest stage I divided everything into English and not-English until we figured out whether it made sense to pursue language ID in general.
The specific blocking tasks do need to be done, but not for all languages at once. For French, Spanish, Italian, and German Wikipedias, we aren't blocked by T121541 specifically, but by the subtask T132466, which is in "needs review", but is basically done.
The language lists for each of those wikis is available in Phab ticket T132466, and that's enough to run the A/B tests parallel to the test we've run for enwiki.
There's still the question of recall-focus vs precision-focus (see T134431 ("needs review", but basically done) and T136034 (to do)), but we can do all the A/B tests with the same precision-focus we've had so far and get a better idea of how well this can work.