Page MenuHomePhabricator
Create Task
Maniphest T342960

Help migrate SDZeroBot to Lift Wing
Open, Needs TriagePublic
Actions

Description

The Wikimedia ML team, are starting the migration of ORES client to another infrastructure, since we are planning to deprecate it. More info in https://wikitech.wikimedia.org/wiki/ORES

The ORES infrastructure is going to be replaced by Lift Wing, a more modern and kubernetes-based service.
All the ORES models (damaging, goodfaith, etc..) are running on Lift Wing, more on how to use them in https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing/Usage
We have new models called Revert Risk, to replace goodfaith and damaging for example. The are available on Lift Wing, and we'd like to offer them as valid and more precise/performant alternative to ORES models. If you'd like to try them we'd help in the migration process!
We have noticed that SDZeroBot is using ORES articlequality and draftquality here and here - Javascript

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenisarantopoulosT342835 Users/UserAgents that use ORES
 OpenNoneT342960 Help migrate SDZeroBot to Lift Wing

Event Timeline

isarantopoulos created this task.Jul 28 2023, 10:57 AM
isarantopoulos moved this task from Unsorted to Backlog/ORES Migration on the Machine-Learning-Team board.Jul 28 2023, 11:03 AM
elukey added a comment.Jul 28 2023, 12:49 PM

https://github.com/siddharthvp/SDZeroBot/issues/34 :)

isarantopoulos updated the task description. (Show Details)Jul 28 2023, 2:42 PM
isarantopoulos added a comment.Sep 27 2023, 2:00 PM

SDZeroBot is now hitting ores-legacy with urls that perform at least 100 LW requests so it fails with 400

isarantopoulos added a comment.Sep 27 2023, 3:06 PM

Created a PR to help tackle the issue. https://github.com/siddharthvp/SDZeroBot/pull/35

SD0001 subscribed.EditedSep 27 2023, 5:28 PM

Have merged the above PR so the 400 errors should be resolved now.

Regarding migration to Lift Wing, how do I request scores for multiple revision ids in a single request? ORES allowed that with a pipe-separated string.

SD0001 added a comment.Sep 27 2023, 5:30 PM

If multiple rev ids in a single call is not possible, what would be a reasonable concurrency level to make LW requests in parallel?

isarantopoulos added a comment.Sep 28 2023, 6:24 AM

Hi @SD0001, thanks for merging that PR! I see that all requests coming from SDZeroBot return a 200 response now.
At the moment you can't request multiple revision ids in the same request but it is something that we're looking into.
Regarding the limit for concurrent requests I don't have an exact number as this may differ from model and wiki, but I will get back to you after we run some tests to support with this migration.

SD0001 added a subscriber: calbon.Sun, May 26, 11:56 AM

Hi @isarantopoulos @calbon ,

With no score caching and requiring the client to make significantly more requests to get the same data, I don't see how this migration has been useful for the end-user so far. The ores-legacy service appears to be slower than the earlier ores service despite giving only 20 scores per request. Ores would provide 50x3=150 scores with lesser latency.

The lack of caching may not be an issue for your internal customers that can implement production-grade subsystems locally. But for external users like Toolforge bots and web services, it would be really useful if LW (or a compatibility layer on top of it) did its own caching and allowed clients to request more scores per call, reducing the load on client resources like the extra threads and network sockets.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL