Page MenuHomePhabricator

Avoid wbgetentities API query in Android app for loading terms
Closed, ResolvedPublic

Description

Currently our infra is heavily under strain due to massive number of requests coming from mobileapps.

Looking in depth shows one reason is that mobileapps loads the whole item via calling wbgetentities API endpoint to load labels and descriptions (https://github.com/wikimedia/apps-android-wikipedia/blob/10f32a8662c12c252cc152348a06674c3087d34d/app/src/main/java/org/wikipedia/dataclient/Service.kt#L360)

This is 1- Extremely slow 2- Have a massive bloat of request response, some items are over MBs of data 3- bypasses four to five layers of cache we put in place for such cases.

It needs to use either https://www.wikidata.org/w/api.php?action=help&modules=query%2Bentityterms or https://www.wikidata.org/w/api.php?action=help&modules=query%2Bpageterms

Event Timeline

Ladsgroup triaged this task as Unbreak Now! priority.Mar 27 2022, 3:50 PM
Krinkle renamed this task from Avoid using wbgetentities for loading item terms to Avoid wbgetentities API query in Android app for loading terms.Mar 27 2022, 4:37 PM

Thanks for the heads up.
Will be fixed via 3225. (to be reviewed / deployed tomorrow)

Looking in depth shows one reason is that mobileapps loads the whole item via calling wbgetentities API endpoint to load labels and descriptions

You mentioned that this is just one reason; Are there other specific API calls that we can be optimizing?

Thanks for the fast response. It didn't turn out as the root cause but one of several issues we found. @Krinkle found some other issues IIRC. The main root cause was some issues in cpjobqueue hitting MCS though.