Page MenuHomePhabricator
Create Task
Maniphest T209459

Wikidata recentchanges API request with ORESscores parameter returns 0 results
Closed, ResolvedPublic
Actions

Description

Reported in https://lists.wikimedia.org/pipermail/wikidata/2018-November/012610.html

Requests like this consistently fail, take lots of time and return 0 results:
https://www.wikidata.org/w/api.php?action=query&format=json&continue=&list=recentchanges&rclimit=500&rcnamespace=0|2|4|6|10|14|120|122|828&rcprop=comment|flags|ids|loginfo|sizes|tags|timestamp|title|user|oresscores
Removing the oresscroes param makes the request respond quickly and with results:
https://www.wikidata.org/w/api.php?action=query&format=json&continue=&list=recentchanges&rclimit=500&rcnamespace=0|2|4|6|10|14|120|122|828&rcprop=comment|flags|ids|loginfo|sizes|tags|timestamp|title|user

Details

SubjectRepoBranchLines +/-
Drop all of service integration with the API hook handlersmediawiki/extensions/ORESmaster+4 -95
Customize query in gerrit

Related Objects

Event Timeline

Addshore created this task.Nov 14 2018, 9:17 AM
Restricted Application added projects: Growth-Team, Machine-Learning-Team. · View Herald TranscriptNov 14 2018, 9:17 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Addshore added a comment.EditedNov 14 2018, 9:20 AM

A profile of the request: https://performance.wikimedia.org/xhgui/run/view?id=5bebe86c3f3dfa882a5a5add

image.png (705×495 px, 38 KB)

Indicates that all of the time is spent in CurlHttpRequest::execute which is called by ORES\ORESService::request

I'm guessing ORES can't respond quickly for requests for 500 scores for recent changes?
Not knowing the details of ORES, this could also be because the scores are not yet pre computed?

Sounds like we need some sort of graceful fallback here.

Addshore renamed this task from Wikidata recentchages API request with oresscores parameter returns 0 results to Wikidata recentchages API request with ORESscores parameter returns 0 results.Nov 14 2018, 9:22 AM
Addshore triaged this task as High priority.
Addshore mentioned this in T209201: WDQS server/updater performance issues.Nov 14 2018, 9:25 AM
Addshore added a comment.EditedNov 14 2018, 9:41 AM

Logs for the request can be found @ https://logstash.wikimedia.org/goto/a1655d3903fbdd2f51a15341bd52dd8c

From the logs:

Requesting: http://ores.discovery.wmnet:8081/v3/scores/wikidatawiki/?models=damaging%7Cgoodfaith&revids=789601229%7C789601232%7C789601231%7C789601230%7C789601228%7C789601227%7C789601226%7C789601224%7C789601225%7C789601223%7C789601222%7C789601220%7C789601221%7C789601219%7C789601218%7C789601217%7C789601215%7C789601211%7C789601214%7C789601213%7C789601212%7C789601206%7C789601208%7C789601209%7C789601207%7C789601205%7C789601203%7C789601200%7C789601195%7C789601198&format=json
Failed to make ORES request to [http://ores.discovery.wmnet:8081/v3/scores/wikidatawiki/?models=damaging%7Cgoodfaith&revids=789601229%7C789601232%7C789601231%7C789601230%7C789601228%7C789601227%7C789601226%7C789601224%7C789601225%7C789601223%7C789601222%7C789601220%7C789601221%7C789601219%7C789601218%7C789601217%7C789601215%7C789601211%7C789601214%7C789601213%7C789601212%7C789601206%7C789601208%7C789601209%7C789601207%7C789601205%7C789601203%7C789601200%7C789601195%7C789601198&format=json], HTTP request timed out.

These requests now happen for me in a matter of ms, so i guess this is due to ORES just not having the results to give to the requestor when requesting them.

Addshore added a project: Performance Issue.Nov 14 2018, 9:46 AM
Ladsgroup claimed this task.Nov 14 2018, 12:58 PM
Ladsgroup edited projects, added Machine-Learning-Team (Active Tasks); removed Machine-Learning-Team.

I will work on it.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptNov 14 2018, 12:58 PM
Addshore moved this task from incoming to in progress on the Wikidata board.Nov 14 2018, 1:17 PM
Jdforrester-WMF subscribed.Nov 14 2018, 4:03 PM
Dinoguy1000 renamed this task from Wikidata recentchages API request with ORESscores parameter returns 0 results to Wikidata recentchanges API request with ORESscores parameter returns 0 results.Nov 14 2018, 6:07 PM
gerritbot subscribed.Nov 15 2018, 3:21 PM

Change 473756 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/ORES@master] Drop all of service integration with the API hook handlers

https://gerrit.wikimedia.org/r/473756

gerritbot added a project: Patch-For-Review.Nov 15 2018, 3:21 PM
Ladsgroup moved this task from Parked to Review on the Machine-Learning-Team (Active Tasks) board.Nov 15 2018, 3:21 PM
gerritbot added a comment.Nov 15 2018, 6:24 PM

Change 473756 merged by jenkins-bot:
[mediawiki/extensions/ORES@master] Drop all of service integration with the API hook handlers

https://gerrit.wikimedia.org/r/473756

Ladsgroup moved this task from Review to Pending deployment on the Machine-Learning-Team (Active Tasks) board.Nov 15 2018, 6:29 PM
ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.6; 2018-11-27).Nov 15 2018, 7:00 PM
Mahir256 mentioned this in T209793: 'rc_comment' column no longer populated in Wikidata's recentchanges table.Nov 18 2018, 7:23 PM
SBisson moved this task from Inbox to External on the Growth-Team board.Nov 19 2018, 1:27 PM
Ladsgroup moved this task from Pending deployment to Completed on the Machine-Learning-Team (Active Tasks) board.Dec 17 2018, 12:46 PM
Lydia_Pintscher subscribed.Jan 4 2019, 2:58 PM

Can this be closed?

Addshore closed this task as Resolved.Jan 4 2019, 3:02 PM

Yup

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL