| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| operations/mediawiki-config | master | +16 -0 | Enable ORES RCFilters for eswikiquote |
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | kostajh | T219160 Enable ORES filters for Spanish Wikiquote | |||
| Resolved | Ladsgroup | T209784 Train/test damaging & goodfaith models for Spanish Wikiquote (eswikiquote) |
Event Timeline
Comment Actions
@SBisson and I met earlier today to review the data from the JSFiddle:
eswikiquote damaging true 0.022 1 precision 0.15 0.152 0.99 eswikiquote damaging true 0.498 1 precision 0.45 0.451 0.797 eswikiquote damaging true 0.705 1 precision 0.6 0.601 0.668 eswikiquote damaging true 0.823 1 precision 0.75 0.752 0.516 eswikiquote damaging true 0.917 1 precision 0.9 0.903 0.339 eswikiquote damaging true 0.962 1 precision 0.98 0.982 0.158 eswikiquote damaging true 0.966 1 precision 0.99 0.992 0.123 eswikiquote damaging true 0.98 1 precision 0.995 1 0.045 eswikiquote damaging true 0.258 1 recall 0.9 0.37 0.901 eswikiquote damaging false 0 0.979 precision 0.15 0.917 1 eswikiquote damaging false 0 0.979 precision 0.45 0.917 1 eswikiquote damaging false 0 0.979 precision 0.6 0.917 1 eswikiquote damaging false 0 0.979 precision 0.75 0.917 1 eswikiquote damaging false 0 0.979 precision 0.9 0.917 1 eswikiquote damaging false 0 0.478 precision 0.98 0.98 0.904 eswikiquote damaging false 0 0.20699999999999996 precision 0.99 0.99 0.835 eswikiquote damaging false 0 0.07599999999999996 precision 0.995 0.995 0.728 eswikiquote damaging false 0 0.46099999999999997 recall 0.9 0.981 0.9 eswikiquote goodfaith true 0 1 precision 0.15 0.936 1 eswikiquote goodfaith true 0 1 precision 0.45 0.936 1 eswikiquote goodfaith true 0 1 precision 0.6 0.936 1 eswikiquote goodfaith true 0 1 precision 0.75 0.936 1 eswikiquote goodfaith true 0 1 precision 0.9 0.936 1 eswikiquote goodfaith true 0.987 1 precision 0.98 0.98 0.964 eswikiquote goodfaith true NaN 1 precision 0.99 NaN NaN eswikiquote goodfaith true NaN 1 precision 0.995 NaN NaN eswikiquote goodfaith true 1 1 recall 0.9 0.985 0.937 eswikiquote goodfaith false 0 0.999 precision 0.15 0.463 0.786 eswikiquote goodfaith false 0 0.999 precision 0.45 0.463 0.786 eswikiquote goodfaith false 0 0.968 precision 0.6 0.601 0.691 eswikiquote goodfaith false 0 0.17000000000000004 precision 0.75 0.753 0.537 eswikiquote goodfaith false 0 NaN precision 0.9 NaN NaN eswikiquote goodfaith false 0 NaN precision 0.98 NaN NaN eswikiquote goodfaith false 0 NaN precision 0.99 NaN NaN eswikiquote goodfaith false 0 NaN precision 0.995 NaN NaN eswikiquote goodfaith false 0 1 recall 0.9 0.064 1
Proposed config
goodfaith
| level | proposed config | precision | recall | range |
|---|---|---|---|---|
| likelygood | maximum recall @precision >= 0.98 | 0.98 | 0.964 | 0.987 - 1 |
| maybebad | maximum recall @precision >= 0.15 | 0.463 | 0.786 | 0 - 0.999 |
| likelybad | maximum recall @precision >= 0.6 | 0.601 | 0.691 | 0 - 0.968 |
| verylikelybad | disabled | n/a | n/a | n/a |
damaging
| level | proposed config | precision | recall | range |
|---|---|---|---|---|
| likelygood | maximum recall @precision >= .99 | .99 | 0.835 | 0 - 0.206 |
| maybebad | maximum filter_rate @recall >= 0.9 | 0.37 | 0.901 | 0.258 - 1 |
| likelybad | maximum recall @precision >= 0.6 | 0.601 | 0.668 | 0.705 - 1 |
| verylikelybad | maximum recall @precision > .98 | 0.982 | 0.158 | 0.962 - 1 |
Comment Actions
Change 500831 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] (wip) Enable ORES RCFilters for eswikiquote
Comment Actions
These look good to me. I'm on the fence as to whether the goodfaith likelybad level should be at precision>=0.6 or precision>=0.75 (as it is, the maybebad and likelybad levels are kind of close together), but for maintainability it might be better to stick with the defaults and let this strangeness sort itself out if/when the model is rebuilt in the future.
Comment Actions
Change 500831 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable ORES RCFilters for eswikiquote
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-04-04T18:23:22Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable ORES RCFilters on eswikiquote (T219160) (duration: 01m 02s)