We first need new tags for draftquality and wp10. This would be as simple as creating an update script to insert the new rows into pagetriage_tags. Then we store the actual ORES values in the pagetriage_page_tags table. The ORES API is easy to work with, and I don't think there are any additional dependencies in order to do this.

We can use the onNewRevisionFromEditComplete to update the ORES data on each edit. Currently we're only using this hook to detect articles created from a redirect, which is somewhat cheap to do. Running the revision through ORES is probably a bit more expensive:

Will it be difficult to re-score these models with every new revision of each page?

I think we need ORES people like @Halfak or someone from performance to tell us whether checking each and every revision, as they are made, is a good idea. Each query can take several seconds to complete, so I have my doubts we can feasibly do this at a high rate. Another option I suppose is to throttle the ORES lookups, so that we only do it once for the same article within say, a 10 minute timeframe, and not for each and every edit. Or, we could use a cronjob that runs every 10 minutes and updates ORES scores for any pages that were edited in the past 10 minutes (the last time the cron ran). The latter sounds easier to me.

I need all pages listed in the New Pages Feed to be filterable with those models, regardless of namespace.

Currently we can only show one namespace at a time, but I don't think it will be a problem to add the option "All" to the namespace dropdown. Hopefully it will be clear this only includes Article, Draft and User namespaces. The filtering otherwise should be easy to do, with two new dropdowns for "Draft quality" and "Assessment".

I need a page's draftquality category and wp10 category to be displayed with its entry in the New Pages Feed list.

Technically it's no problem. For draft quality, I'd suggest only showing the overall prediction and it's score, so "OK (65%)" or "attack (95%)". Similar for wp10, with "Stub (85%)" or "FA (60%)" (as if there were ever any newly created FAs :). This would keep it concise, and should be able to fit nicely within the "55 bytes · 1 edit · 1 category" region.

I need to be able to sort the New Pages Feed by the scores underlying the categories of draftquality and wp10.

I don't see why we couldn't do this, technically. We'd add two new options at the "Sort by:" list at the top-right, "Draft quality" and "WP10", each with a dropdown.

What will happen when we score these models on the User namespace, which is currently accessible in the New Pages Feed?

I probably wouldn't do this. Yes, there are drafts created in the userspace, but there are probably just as many "Hi my name is Bob" user pages. I did some quick testing, and it seems like ORES does a good job of detecting that these are at least good-faith, but the scoring in general probably isn't that helpful for the purposes of reviewing. Maybe we could do it only for submitted drafts in the userspace (which by the way are often moved by the reviewer to Draft)? I just want to weigh out the benefit of scoring the userspace versus performance impact.

Aside from the above, I think the remaining user stories can be fulfilled. I will wait for further feedback before creating any new tickets.

@MusikAnimal -- thanks for the investigating so far. Though some of what you said is beyond my technical depth, I have some responses and clarifications:

• For the story that reads "I need all pages listed in the New Pages Feed to be filterable with those models, regardless of namespace." -- I actually just meant that the feed should be filterable by those models no matter which namespace you've selected (i.e. we score all the relevant namespaces with the models). We don't ever want all the namespaces to be in the same list at the same time, because that would make it easy for a reviewer to accidentally apply the wrong criteria to a page they're reviewing. I'll clarify this in the description.
• I like the idea of listing the score in parentheses next to the category (@alexhollender).
• We've since decided we don't actually have this story anymore, and I'll clarify in the description: "I need to be able to sort the New Pages Feed by the scores underlying the categories of draftquality and wp10."

In T193807#4198661, @MusikAnimal wrote:

whether checking each and every revision, as they are made, is a good idea.

It would only be every revision to a page that is in the AFC pending list though wouldn't it? (Which is defined by category membership.) So it'd slow down saving of those pages, but maybe that's acceptable? Or if that's no good, it could only calculate the scores when being added to the list.

In T193807#4198930, @Samwilson wrote:

In T193807#4198661, @MusikAnimal wrote:

whether checking each and every revision, as they are made, is a good idea.

It would only be every revision to a page that is in the AFC pending list though wouldn't it? (Which is defined by category membership.) So it'd slow down saving of those pages, but maybe that's acceptable? Or if that's no good, it could only calculate the scores when being added to the list.

I just assumed we were doing it for all pages in Special:NewPagesFeed, or at least that would be useful if we did. I think the updates to tags are deferred, so shouldn't noticeably slow anything down (disregard, not how deferred updates work, apparently!)

Just to wrap up this investigation:

• There are two main options to do this:
  • Use the ORES API ourselves, and store the scores in the pagetriage_page_tags table in the page triage database.
  • Use the table in the Mediawiki database that Scoring team is already storing scores in. This relies on the Scoring team's timelines to get the draftquality and wp10 models into that table.
• We prefer the latter option, provided that it is ready on the timelines that this project will work on.

To continue to follow along with this element of the project, see T195796.