Page MenuHomePhabricator

Some RC changes do not display highlight with ORES Quality and Intent filters
Open, Needs TriagePublic

Description

The issue is most noticeable in enwiki.

Go to RC page and select to highlight (without not selecting filters) all 'Contribution quality prediction filters' - Very likely good, May have problems, Likely have problems, and Very likely have problems.

There will be changes that are not highlighted, e.g.

Screen Shot 2017-08-08 at 4.29.12 PM.png (693×1 px, 251 KB)
Those non-highlighted changes are of various types (Category, tagged as Mobile edits, anon edits, and bot edits). I checked normal looking article edits, e.g. the following did not display any highlighting (see the screenshot above):
(diff | hist) . . Portland Union Station‎; 23:25 . . (+1)‎ . . ‎184.100.183.206 (talk)‎ (→‎Transit services) (Tags: Mobile edit, Mobile web edit)

https://en.wikipedia.org/w/index.php?title=Portland_Union_Station&curid=1026735&diff=794602613&oldid=794602468

[enwiki]> select * from ores_classification where oresc_rev=794602468 ;
+-----------+-----------+-------------+-------------+-------------------+--------------------+
| oresc_id  | oresc_rev | oresc_model | oresc_class | oresc_probability | oresc_is_predicted |
+-----------+-----------+-------------+-------------+-------------------+--------------------+
| 145153117 | 794602468 |          21 |           1 |             0.574 |                  1 |
| 145153118 | 794602468 |          22 |           1 |             0.277 |                  0 |
+-----------+-----------+-------------+-------------+-------------------+--------------------+
2 rows in set (0.00 sec)

@jmatazzoni here: Just to clarify @Etonkovidova's points above: the following screenshot shows me filtering for human-created page edits to articles. All such edits should get ORES scores. But when I apply one color for each of the four ORES Quality categories, many edits still don't have colors. This should be impossible, since the ORES Quality categories cover all possible scores, from 0 to 100.

Screen Shot 2017-08-08 at 4.35.55 PM.png (793×1 px, 277 KB)

I made a similar test with the User Intent filters, with similar results. See screenshot.

Screen Shot 2017-08-08 at 4.40.22 PM.png (769×1 px, 291 KB)

Event Timeline

jmatazzoni renamed this task from Some RC changes do not display highlight with ORES Contribution quality filters to Some RC changes do not display highlight with ORES Quality and Intent filters .Aug 8 2017, 11:41 PM
jmatazzoni updated the task description. (Show Details)
jmatazzoni subscribed.

I looked into a few of these cases and none of these edits have entries in the ores_classification table even though they should. So it looks like ORES occasionally craps out or times out and a few edits go unscored.