Page MenuHomePhabricator

New Pages Feed: list ORES categories in the feed (3.4)
Closed, ResolvedPublic

Description

This is a task for a user story that was initially scoped out of the New Pages Feed project, but now seems to be easier than we initially thought. Therefore, I am creating the task so that we can attempt it. This is the user stories:

  • As a reviewer, I need to see a page's draftquality category and wp10 category along with its listing in the New Pages Feed.

Note that this is similar work to T199155, which is about showing a draft's AfC state and dates with its listing.

Specifically, when either "New Page Patrol" or "Articles for Creation" is selected for the New Pages Feed:

  • Add an element to the listing of drafts in the New Pages Feed that says the draft's draftquality category along with the preface "Predicted issues:". The engineer can decide where in the listing it makes most sense to put this. These are the possible values:
    • "Predicted issues: none"
    • "Predicted issues: spam"
    • "Predicted issues: vandalism"
    • "Predicted issues: attack"
  • Add an element to the listing of drafts in the New Pages Feed that says the draft's wp10 category along with the preface "Predicted class:". The engineer can decide where in the listing it makes most sense to put this. These are the possible values:
    • "Predicted class: Stub"
    • "Predicted class: Start"
    • "Predicted class: C-class"
    • "Predicted class: B-class"
    • "Predicted class: Good"
    • "Predicted class: Featured"
  • The above changes should exist for all pages in the feed, regardless of whether they are in main, user, or draft space.

Event Timeline

MMiller_WMF moved this task from Inbox to Q1 2018-19 on the Growth-Team board.Jul 10 2018, 5:35 PM

@kostajh -- below is an early mockup of what the listing in the feed might look like with additional elements added. You can use your own judgment on how to arrange it, but this could provide some inspiration.

MaxSem removed a subscriber: MaxSem.Jul 16 2018, 6:27 PM
Niharika removed a subscriber: Niharika.Jul 16 2018, 7:36 PM

@MMiller_WMF can you move this to this sprint? I do not have the capability.

kostajh triaged this task as Normal priority.Jul 18 2018, 9:36 PM
SBisson claimed this task.Aug 1 2018, 2:34 PM
SBisson added subscribers: Halfak, SBisson.

I just noticed from the task description:

The above changes should exist for all pages in the feed, regardless of whether they are in main, user, or draft space.

The ORES scores in question (wp10, draftquality) are NOT currently available for the user namespace. If it's important, we can check with @Halfak if they can be enabled.

Indeed. This was to limit the size of the prediction table in MW. It seems that scores don't make sense (and could be very wrong) for *most* user pages. Only pages that are intended to be treated as article drafts would be relevant. Is there some reasonable way that we can maintain scores for only those pages? If not, maybe we can just maintain scores for all user pages anyway because the data should not be that big. @Ladsgroup, what do you think?

Indeed. This was to limit the size of the prediction table in MW. It seems that scores don't make sense (and could be very wrong) for *most* user pages. Only pages that are intended to be treated as article drafts would be relevant. Is there some reasonable way that we can maintain scores for only those pages? If not, maybe we can just maintain scores for all user pages anyway because the data should not be that big. @Ladsgroup, what do you think?

We can trigger the scoring only for user pages that are of interest to PageTriage. That would limit the size. We've worked on something similar for T199357: New Pages Feed: score draftquality on most recent revision

We can trigger the scoring only for user pages that are of interest to PageTriage. That would limit the size. We've worked on something similar for T199357: New Pages Feed: score draftquality on most recent revision

Yes, I would recommend moving towards that. Keep the logic of PageTriage inside the extension.

@SBisson @Ladsgroup @Halfak -- it is okay for us not to have those scores for the User namespace. I do not think people will need or use them. We don't need to bother with triggering the scoring, unless it is exceedingly easy. If the reviewers come back and tell us they want them, then we could revisit in the future.

@SBisson @Ladsgroup @Halfak -- it is okay for us not to have those scores for the User namespace. I do not think people will need or use them. We don't need to bother with triggering the scoring, unless it is exceedingly easy. If the reviewers come back and tell us they want them, then we could revisit in the future.

It is in fact so easy that I think I've done it by accident for 'draftquality' in https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/PageTriage/+/449275/

I need to add a few lines to a) prevent it or b) do it for 'wp10' as well. We can go either way. I would prefer to add it so we don't have to deal with a bunch of "Score: N/A" in the UI for the user namespace.

On the one hand, it doesn't make sense for a lot of User pages to have wp10 score because they are about the user. But then there are the draft articles that are created in User space, for whom it does make sense.

But if the easiest thing is to display them, let's do it, and then we can remove it later if the reviewers say that it is confusing.

Change 447103 had a related patch set uploaded (by Sbisson; owner: Kosta Harlan):
[mediawiki/extensions/PageTriage@master] Add ORES and AfC information to list items

https://gerrit.wikimedia.org/r/447103

Change 447103 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Add ORES and AfC information to list items

https://gerrit.wikimedia.org/r/447103

ORES models draftquality and wp10 with respective classes are displayed as filter options and work as expected.

(1) In betalabs there are many article revisions that will show` Predicted class: N/A Predicted issues: N/A` (the same is true for Drafts also). Do we expect to have some revisions not to be scored in testwiki and other production wikis? If we do, we need to be ready to explain why some revisions do not get scored.

(2) User namespace content is a subject to NPP and Review (AFCH) - per comment https://phabricator.wikimedia.org/T201098#4478171:

The special case is userspace since those pages can be in both queues and patrolling those should have a different effect on each queue.

Those two processes do not affect each other much - except when it comes to Declined-Resubmit. After Resubmit, the 'Review waiting' template is displayed on a page but from the point of view of NPP, the page is still reviewed( NPP Reviewed filter).
'Review (AFCH)' is disabled on Article pages and 'Review' patrol toolbar is disabled for Draft namespace.

This all looks good. Moving to Done.

MMiller_WMF closed this task as Resolved.Aug 16 2018, 8:05 PM