Page MenuHomePhabricator

Create a mirror for the Mediawiki API to allow faster requests
Closed, DeclinedPublic

Description

During the hachathon 2017, @PierreSelim gave me this feedback. He was comparing the analytics API for stats and the classical Mediawiki API. "Make a request on Mediawiki API is very slow, because the API is requesting directly on the wiki. Thresholds are very low to request the wiki. Some big requests are very slow and would go faster if the API calls were possible from a mirror site without any risk to put the wiki down."

Event Timeline

Can you elaborate? I don't understand why a mirror would necessarily be quicker

Anomie closed this task as Declined.EditedMay 19 2017, 3:37 PM
Anomie subscribed.

It's not clear to me exactly what is being requested here, but either way it looks like the answer is "no".

If you're talking about individual requests being "slow", a mirror isn't going to help that. In fact, since mirror infrastructure would likely be underresourced compared to the production infrastructure, using mirror infrastructure would likely be even slower. Further, I haven't heard anything that makes me believe that production infrastructure cannot handle the level of API queries we receive, but if that were the case it's more likely that the hardware would be better spent on expanding the production infrastructure instead of creating a separate mirror infrastructure.

If, on the other hand, you're talking about "queries" that take many individual API requests thanks to limits in place, such as fetching a list of all the pages in the English Wikipedia's Category:Living people, then a mirror infrastructure running MediaWiki but with higher limits is not the answer. You'll want to look at the database replicas available in Tool Labs, or at loading database dumps into your own infrastructure.

Thank you for the ideas, @Anomie. I'll share them.