Based on the documentation here: https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_apps#Routes
I would expect to find MCS running for WikiSpecies here: https://species.wikipedia.org/api/rest_v1/page/mobile-sections/Main_Page
Just like it does for EN: https://en.wikipedia.org/api/rest_v1/page/mobile-sections/Main_Page
Does MCS run for species? If so, what is the url to use?
Issue for MWOffliner: https://github.com/openzim/mwoffliner/issues/767
Yes, we could, but likely with an ETA of 2 months. When you say MCS, does that include all of MCS-backed end points?
I think we'd like this is for all MCS/PCS endpoints (mobile-sections, mobile-html, and related PCS JSON and CSS/JS endpoints).
@mobrovac We're only actually using mobile-sections in MWOffliner. So the mobile-sections endpoint is our "priority" :)
I am aware of that, but we have to strive for consistency across our projects. And actually, technically speaking, allowing only mobile-sections for species.wm.org would take more time than enabling all MCS-backed end points :)
The reason why I mentioned mobile-html and the other PCS endpoints is that those will replace mobile-sections in the future.
BTW, it.wikibooks.org seems to be also impacted. Do we have a list somewhere of all the wikis where the mobile API endpoints are not "properly" available?
The list of domains and their config type can be found here. All domains that have the default_project type associated to them don't expose the mobile APIs. That invcludes all wikibooks, wikinews, wikiversities, wikiquotes and wikisources.
@mobrovac Thx, so I see that this concerns 307 web sites and for us, as around 20 are meta web sites that we don't want to scrape anyway, this is around 280 wikis we hardly can scrape.
$ curl "https://github.com/wikimedia/mediawiki-services-restbase-deploy/blob/4b34370ea3f80ac0995333b756897e158d4f5a3d/scap/vars.yaml" 2> /dev/null | grep default_project | wc 307 2763 51494
The mobile end points are purpose-built and targeted towards specific projects. Unfortunately, due to the very different formatting conventions used by various projects, we can't guarantee meaningful results for all projects. @bearND and @Mholloway can comment on the specifics.
The mobile-sections endpoint and most of the other MCS/PCS endpoints were built to support the Wikipedia apps. So, most endpoints have only been tested with *.wikipedia.org domains. (The exception is the definitions endpoint but you probably don't care about that one.)
May I ask what features of mobile-sections you are most interested in and are the reason for not using Parsoid directly?