⚓ T314302 ORES doesn't support fawiki family template

Status	Assigned	Task
Resolved	kevinbazira	T314302 ORES doesn't support fawiki family template
Resolved	kevinbazira	T317531 Retrain fawiki articlequality model
Resolved	kevinbazira	T319373 Deploy new fawiki articlequality model to ORES and LiftWing
Resolved	kevinbazira	T322614 Update existing fawiki-articlequality isvc with new model on LiftWing

calbon created this task.Aug 1 2022, 2:23 PM

Restricted Application added subscribers: Huji, Aklapper. · View Herald TranscriptAug 1 2022, 2:23 PM

Thid is an example of ref-->sfn conversion which lowers the ORES score ---> https://fa.wikipedia.org/w/index.php?title=زوسیموس&diff=35130948&oldid=35130784&diffmode=source

kevinbazira moved this task from Parked to In Progress on the Machine-Learning-Team (Active Tasks) board.Aug 2 2022, 1:28 PM

ORES models are built using a library called revscoring. Based on the way revscoring processes wikitext using wikitext parsed features, both ref wikitext tags and sfn wikitext templates should be taken care of in this section that lists ref_tags and templates.

I run a quick test using the fawiki ediquality goodfaith model comparing revid 35130784 with 35130948 and below are the results:

35130784 with ref tag returns {'prediction': True, 'probability': {False: 0.13646938691972665, True: 0.8635306130802733}}
35130948 with sfn template returns {'prediction': True, 'probability': {False: 0.02206572256478212, True: 0.9779342774352179}}

There is a slight change in the probability figures but this didn't overturn the prediction.

kevinbazira mentioned this in rOWCbb9e711565d7: feature-lists: Add sfn templates to fawiki features.Sep 8 2022, 9:29 AM

kevinbazira mentioned this in T317531: Retrain fawiki articlequality model.Sep 12 2022, 12:24 PM

kevinbazira mentioned this in rOWC55bc0f8fefcc: Retrain fawiki articlequality model.Oct 4 2022, 1:16 PM

calbon moved this task from Active Tasks to In Progress on the Machine-Learning-Team board.Oct 25 2022, 6:23 PM

calbon edited projects, added Machine-Learning-Team; removed Machine-Learning-Team (Active Tasks).

Support for sfn templates was added to fawiki articlequality, the model was retrained (T317531), evaluated (T317531#8362584), and deployed (T319373) to ORES and LiftWing.

@Mojtaba-kd, please let us know in case this issue persists. Thanks!