- Deploy models to beta and smoke test.
- Deploy to production.
- Enable new models via configuration.
- Announce on-wiki.
Description
Details
Subject | Repo | Branch | Lines +/- | |
---|---|---|---|---|
Enable ORES on cawiki | operations/mediawiki-config | master | +10 -0 |
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | Catrope | T192501 Deploy ORES advanced editquality models to cawiki | |||
Resolved | Ladsgroup | T187732 Train/test damaging/goodfaith model for Catalan Wikipedia |
Event Timeline
@Townie Would you be willing to draft a release announcement for these upcoming models? We expect to have the deployment ready by the end of the month.
Here's an example announcement: https://tr.wikipedia.org/w/index.php?title=Vikipedi:K%C3%B6y_%C3%A7e%C5%9Fmesi_(ilginize)&oldid=17339200#ORES_review_tool_deployment_in_Turkish_Wikipedia
Wonderful, thank you! We'll post more information here as the deployment goes forward.
@Trizek-WMF, heads-up that this announcement has just been published on cawiki, see the previous comment.
Change 431036 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] Enable ORES on cawiki
Note that the goodfaith model for cawiki isn't super helpful. There's a quite good "may be bad faith" filter with 32% precision and 89.1% recall, but the threshold range for that is 0.000 through 0.999. So the only way to make "very likely good faith" not overlap with "may be bad faith" was to set its threshold range to be 1 through 1 (i.e. just the value 1), which has 99.8% precision and 97.2% recall. This seems to indicate that a very large proportion of all inputs get score 1.
Using the default settings would have pegged "very likely good faith" at 99.5% precision, which has 99.5% recall and corresponds to a threshold range of 0.883 through 1. This would have overlapped both with "may be bad faith" (0 - 0.999 at 32/89) and "likely bad faith" (0 - 0.926 at 60/70). On the other end of the spectrum, the "very likely bad faith" filter gets 90.4% precision with 29.2% recall, but its threshold range is 0 - 0.008.
Maybe this is all fine and the model is just very confident, but I felt a bit dirty setting this up.
Change 431036 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable ORES on cawiki
Mentioned in SAL (#wikimedia-operations) [2018-05-09T23:53:18Z] <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Enable ORES on cawiki, lvwiki, huwiki (T192501, T192499, T192496) (duration: 01m 29s)