Page MenuHomePhabricator

Include specific user groups in the trwiki edit quality model
Closed, ResolvedPublic

Description

@Superyetkin suggested that some user groups should be whitelisted in the edit quality predictions. List them out, include a feature in the models for them and retrain.

Trwiki specific features live here: https://github.com/wiki-ai/editquality/blob/master/editquality/feature_lists/trwiki.py

We gave a general set of features for user-rights here: https://github.com/wiki-ai/editquality/blob/master/editquality/feature_lists/mediawiki.py#L16

You'll likely want to replicate this pattern inside of the trwiki.py file for the specific "whitelist" user and then retrain the model. To do that, log into ores-compute-01.eqiad.wmflabs and rebuild the trwiki models.

Event Timeline

Halfak created this task.Jul 15 2016, 3:07 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 15 2016, 3:07 PM

@Superyetkin, what were the user groups who you think should be excluded?

Halfak triaged this task as Normal priority.Jul 15 2016, 3:08 PM
Halfak moved this task from Untriaged to New development on the Scoring-platform-team board.
Halfak moved this task from New development to Maintenance/cleanup on the Scoring-platform-team board.
Superyetkin added a comment.EditedJul 15 2016, 5:14 PM

what were the user groups who you think should be excluded?

"whitelist" group (and those that have the derived privileges) should be excluded.

Halfak updated the task description. (Show Details)Jul 26 2016, 2:32 PM
Halfak assigned this task to schana.

Is there anyone who can rebuild the model for trwiki?

Yup. We're currently bogged down a bit by some operations work. But we have assigned this task and we should be able to pick it up in the next week. Sorry for the delay and thanks for your patience and collaboration.

Halfak reassigned this task from schana to Ladsgroup.Aug 16 2016, 4:41 PM
Halfak added a subscriber: schana.
Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptAug 16 2016, 4:41 PM

@Superyetkin: Hey, I'm going to pick this up and do it. I checked and trwiki doesn't have "whitelist" group neither the right. here's list of them. Can you point out to me what user group do you mean? I can think of autoreview and patroller. Do you mean those?

Yes, the correct group name is "autoreview". The change would also work for the "patroller" group. Thanks.

Ladsgroup moved this task from In progress to Done on the User-Ladsgroup board.Aug 19 2016, 2:19 PM
Ladsgroup closed this task as Resolved.Aug 22 2016, 6:22 PM
Superyetkin reopened this task as Open.Aug 22 2016, 7:05 PM

We still see some edits made by users with "autoreview" rights being flagged as harmful. Here is an example.

Ladsgroup added a comment.EditedAug 22 2016, 7:13 PM

It's fixed but it's not deployed yet. We will deploy it possibly this week and also bear in mind we are improving our system to know the rights and possibly don't show the "r" flag for them but we can't eliminate all cases. It's AI after all.

Halfak closed this task as Resolved.Aug 29 2016, 10:55 PM