Page MenuHomePhabricator

Increased errors in GET https://ar.wikipedia.org/api/rest_v1/page/random/summary since 21 Jan 9:00 UTC
Closed, ResolvedPublic

Description

We have observed a high error count in GET https://ar.wikipedia.org/api/rest_v1/page/random/summary since 21 Jan 9:00 UTC

https://logstash.wikimedia.org/goto/bf811f2231648f2c306371b8c4f6e343

arwiki.png (398×1 px, 59 KB)

The endpoint looks very slow indeed:

jynus@sangai:/media/home/jynus/puppet$ curl -o /dev/null -s -w 'Total: %{time_total}s\n' https://ar.wikipedia.org/api/rest_v1/page/random/summary
Total: 16.813162s
jynus@sangai:/media/home/jynus/puppet$ curl -o /dev/null -s -w 'Total: %{time_total}s\n' https://en.wikipedia.org/api/rest_v1/page/random/summary
Total: 0.485580s
jynus@sangai:/media/home/jynus/puppet$ curl -o /dev/null -s -w 'Total: %{time_total}s\n' https://ar.wikipedia.org/api/rest_v1/page/random/summary
Total: 26.566031s
jynus@sangai:/media/home/jynus/puppet$ curl -o /dev/null -s -w 'Total: %{time_total}s\n' https://en.wikipedia.org/api/rest_v1/page/random/summary
Total: 0.481798s

Event Timeline

jijiki triaged this task as High priority.Jan 28 2020, 12:17 PM
Krinkle subscribed.

Tagging Parsing and CPT which are behind the actual endpoint. Not sure whether this endpoint is from Parsoid or core REST (or a one on top of the other). Feel free to retag/untag accordingly.

Anomie subscribed.

MediaWiki-REST-API is about /w/rest.php; this task appears to be about something from Mobile-Content-Service (behind Restbase/Restrouter) instead.

Thanks @Anomie, I misread the url. Thought it was /w/rest.php!

Change 569045 had a related patch set uploaded (by Mholloway; owner: Michael Holloway):
[mediawiki/services/mobileapps@master] Do not request text extracts in lib/random.js

https://gerrit.wikimedia.org/r/569045

I still don't have a good theory about what changed to cause problems, but I've just submitted a patch that should help to reduce the load on the appservers from the API request associated with the /page/random PCS endpoint.

Change 569045 merged by jenkins-bot:
[mediawiki/services/mobileapps@master] Do not request text extracts in lib/random.js

https://gerrit.wikimedia.org/r/569045

Just deployed the change, which fixed the performance problem on arwiki. It seems that TextExtracts extract generation is very slow for Arabic. That's now removed:

mholloway@mholloway:~/code/wikimedia/mobileapps$ curl -o /dev/null -s -w 'Total: %{time_total}s\n' https://ar.wikipedia.org/api/rest_v1/page/random/summary
Total: 0.498373s
Mholloway claimed this task.