Page MenuHomePhabricator

Enable ORES filters for Spanish Wikiquote
Closed, ResolvedPublic

Details

Related Gerrit Patches:
operations/mediawiki-config : masterEnable ORES RCFilters for eswikiquote

Event Timeline

Halfak created this task.Mar 25 2019, 2:14 PM
Restricted Application added a project: artificial-intelligence. · View Herald TranscriptMar 25 2019, 2:14 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Harej added a subscriber: Harej.Mar 26 2019, 2:06 AM

@Halfak Please clarify whether this ticket depends on work by Scoring Platform.

It does not. This is work that the Growth-Team performs.

We will work on this in our next sprint.

kostajh claimed this task.Apr 1 2019, 7:24 PM
kostajh edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
kostajh moved this task from Incoming to In Progress on the Growth-Team (Current Sprint) board.
kostajh added a subscriber: SBisson.Apr 2 2019, 8:16 PM

@SBisson and I met earlier today to review the data from the JSFiddle:

eswikiquote	damaging	true	0.022	1	precision 0.15	0.152	0.99
eswikiquote	damaging	true	0.498	1	precision 0.45	0.451	0.797
eswikiquote	damaging	true	0.705	1	precision 0.6	0.601	0.668
eswikiquote	damaging	true	0.823	1	precision 0.75	0.752	0.516
eswikiquote	damaging	true	0.917	1	precision 0.9	0.903	0.339
eswikiquote	damaging	true	0.962	1	precision 0.98	0.982	0.158
eswikiquote	damaging	true	0.966	1	precision 0.99	0.992	0.123
eswikiquote	damaging	true	0.98	1	precision 0.995	1	0.045
eswikiquote	damaging	true	0.258	1	recall 0.9	0.37	0.901
eswikiquote	damaging	false	0	0.979	precision 0.15	0.917	1
eswikiquote	damaging	false	0	0.979	precision 0.45	0.917	1
eswikiquote	damaging	false	0	0.979	precision 0.6	0.917	1
eswikiquote	damaging	false	0	0.979	precision 0.75	0.917	1
eswikiquote	damaging	false	0	0.979	precision 0.9	0.917	1
eswikiquote	damaging	false	0	0.478	precision 0.98	0.98	0.904
eswikiquote	damaging	false	0	0.20699999999999996	precision 0.99	0.99	0.835
eswikiquote	damaging	false	0	0.07599999999999996	precision 0.995	0.995	0.728
eswikiquote	damaging	false	0	0.46099999999999997	recall 0.9	0.981	0.9
eswikiquote	goodfaith	true	0	1	precision 0.15	0.936	1
eswikiquote	goodfaith	true	0	1	precision 0.45	0.936	1
eswikiquote	goodfaith	true	0	1	precision 0.6	0.936	1
eswikiquote	goodfaith	true	0	1	precision 0.75	0.936	1
eswikiquote	goodfaith	true	0	1	precision 0.9	0.936	1
eswikiquote	goodfaith	true	0.987	1	precision 0.98	0.98	0.964
eswikiquote	goodfaith	true	NaN	1	precision 0.99	NaN	NaN
eswikiquote	goodfaith	true	NaN	1	precision 0.995	NaN	NaN
eswikiquote	goodfaith	true	1	1	recall 0.9	0.985	0.937
eswikiquote	goodfaith	false	0	0.999	precision 0.15	0.463	0.786
eswikiquote	goodfaith	false	0	0.999	precision 0.45	0.463	0.786
eswikiquote	goodfaith	false	0	0.968	precision 0.6	0.601	0.691
eswikiquote	goodfaith	false	0	0.17000000000000004	precision 0.75	0.753	0.537
eswikiquote	goodfaith	false	0	NaN	precision 0.9	NaN	NaN
eswikiquote	goodfaith	false	0	NaN	precision 0.98	NaN	NaN
eswikiquote	goodfaith	false	0	NaN	precision 0.99	NaN	NaN
eswikiquote	goodfaith	false	0	NaN	precision 0.995	NaN	NaN
eswikiquote	goodfaith	false	0	1	recall 0.9	0.064	1

Proposed config

goodfaith

levelproposed configprecisionrecallrange
likelygoodmaximum recall @precision >= 0.980.980.9640.987 - 1
maybebadmaximum recall @precision >= 0.150.4630.7860 - 0.999
likelybadmaximum recall @precision >= 0.60.6010.6910 - 0.968
verylikelybaddisabledn/an/an/a

damaging

levelproposed configprecisionrecallrange
likelygoodmaximum recall @precision >= .99.990.8350 - 0.206
maybebadmaximum filter_rate @recall >= 0.90.370.9010.258 - 1
likelybadmaximum recall @precision >= 0.60.6010.6680.705 - 1
verylikelybadmaximum recall @precision > .980.9820.1580.962 - 1

Change 500831 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[operations/mediawiki-config@master] (wip) Enable ORES RCFilters for eswikiquote

https://gerrit.wikimedia.org/r/500831

Catrope added a subscriber: Catrope.Apr 2 2019, 8:54 PM

These look good to me. I'm on the fence as to whether the goodfaith likelybad level should be at precision>=0.6 or precision>=0.75 (as it is, the maybebad and likelybad levels are kind of close together), but for maintainability it might be better to stick with the defaults and let this strangeness sort itself out if/when the model is rebuilt in the future.

Change 500831 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable ORES RCFilters for eswikiquote

https://gerrit.wikimedia.org/r/500831

Mentioned in SAL (#wikimedia-operations) [2019-04-04T18:23:22Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable ORES RCFilters on eswikiquote (T219160) (duration: 01m 02s)

Thank you very much to everyone!

Etonkovidova closed this task as Resolved.Apr 10 2019, 8:05 PM