Note, we're working to enable the ORES extension on testwiki.wikipedia.org (T199913).

if @Halfak signs this off, I make it happen. Basically the question boils down to how we produce random homogeneous scores for a multi-class classifier.

+1 for enabling. I made a change in T199239: Implement named aliases in ScorerModels that allows us to safely (and efficiently) configure models to be used across contexts.

@Ladsgroup, make sure that you use the same model key (e.g. enwiki_draftquality and enwiki_wp10) in the configuration for the scoring context of testwiki. Otherwise, everything should work fine :)

Relatedly, we can do this for the damaging/goodfaith models as well. The result would be that testwiki edits are actually scored by the enwiki models rather than using a deterministic scoring strategy. I'm not sure if that would be useful or if it would disrupt any ongoing work.

I'm not sure about this approach, It's good for production but the task description asks about having wp10 testwiki model in vagrant which we don't have access to enwiki models there and we can only go with deterministic approaches. I like it too because of better decoupling (if I want to test wp10 model in my localhost, I should not need to pull in editquality repo)

Not sure this is the right task to bring it up but we would really like to have real scoring for 'wp10' and 'draftquality' on test.wikipedia.org. We plan on doing user testing of the PageTRiage/ORES integration on that site.

Using the models trained on enwiki production sounds reasonable since article content would likely be imported from there.

Yes, thanks @SBisson. It is definitely important for us to be able to test that the right scores are being populated to the New Pages Feed as part of the work in T193782. The reason this is particularly relevant for this work is because we want to have actual English Wikipedia users testing this feature in test wiki, and it will make a big difference if the scores they see are realistic.

Change 450392 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/services/ores/deploy@master] Add wp10 and draftquality for testwiki

https://gerrit.wikimedia.org/r/450392

Change 450392 merged by Halfak:
[mediawiki/services/ores/deploy@master] Add wp10 and draftquality for testwiki

https://gerrit.wikimedia.org/r/450392

Change 451222 had a related patch set uploaded (by Catrope; owner: Catrope):
[operations/mediawiki-config@master] Enable wp10 and draftquality ORES models on testwiki

https://gerrit.wikimedia.org/r/451222

In T198997#4481811, @gerritbot wrote:

Change 450392 merged by Halfak:
[mediawiki/services/ores/deploy@master] Add wp10 and draftquality for testwiki

https://gerrit.wikimedia.org/r/450392

@Ladsgroup Is this scheduled to be deployed during tomorrow's ORES window at 20:00 UTC? If so, we could deploy the above config patch in the same window, or in the following SWAT window at 23:00 UTC.

@awight is working on come updates to some of our models. I believe that is the only blocker to the next ORES deployment. I've just created T201518: ORES deployment (Early August) to track the changes that will go out with this deployment.

Change 451222 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable wp10 and draftquality ORES models on testwiki

https://gerrit.wikimedia.org/r/451222

Mentioned in SAL (#wikimedia-operations) [2018-08-13T23:13:01Z] <catrope@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Enable wp10 and draftquality ORES models on testwiki (T198997) (duration: 00m 51s)