Page MenuHomePhabricator

Check ApiQueryReferences compatibility with extended references
Closed, ResolvedPublic2 Estimated Story Points

Description

The Cite codebase contains a query API module for the MediaWiki action API. Example API call: https://en.wikipedia.org/wiki/Special:ApiSandbox#action=query&prop=references.

It appears like we (currently) don't need to do anything about this, because this API appears to be disabled on all Wikimedia wikis. The flag $wgCiteStoreReferencesData is false by default and never enabled, as far as I can tell.

To do:

  • Check if the API is enabled anywhere, and where.
  • Even we can't find any usage, we still might break usages we are not aware of. Check how the API behaves when confronted with extended references, and decide on action items.

Event Timeline

Restricted Application added a project: Platform Engineering. · View Herald TranscriptNov 13 2019, 10:42 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Anomie added a subscriber: Anomie.

Background: The module was added in rECIT509741dc1714: Surface references via api query property, with reference to T123290: Should be possible to access references and notes for a given page via API. That task was filed as a subtask of T123328: [GOAL] Lazy load references in mobile skin, which was later declined and the beta implementation removed (T222373: Remove the lazy load references beta feature).

I see no evidence that $wgCiteStoreReferencesData was ever enabled in Wikimedia production or on Beta Cluster. This may be due to discussion starting at T125329#2003930, which raised many concerns over the existing implementation.

It's possible that the API module and $wgCiteStoreReferencesData should be deprecated and removed, particularly if it would otherwise take significant work to make it work with extended references.

Since https://www.mediawiki.org/wiki/Developers/Maintainers lists Editing-team as the owner of the extension, I'll tag them in case they have further input.

Wow, that's an incredibly helpful response. Thanks a lot!

I tested the API locally. This is how the relevant part looks like:

"references": {
	"cite_note-first-1": {
		"text": "First note, declared somewhere in the article",
		"count": 1,
		"key": 1,
		"number": 1,
		"dir": null,
		"name": "first",
		"group": "",
		"reflist": 0
	},

Note the text is unparsed Wikitext – heavily limiting the usefulness of this API, I suspect. For comparison: the REST API returns parsed HTML.

It looks like it might be easy to add the extends attribute. We might even get this for free, because what the API does is nothing but exposing the private (!) Cite::$mRefs property.

However. A relevant part of our project is to clean the Cite codebase up as good as we can, within our time and budget constrains. If we can just kill this API, that would be great! It's a lot of code: The API class (~210 lines), two hook handlers (~70 lines), and even something in the Cite class itself (~40 lines). Not having to deal with this and all the constrains, dependencies, and implications it comes with would save us a lot of time.

I've additionally verified that virtually no API requests come in for this endpoint. There are one or two per day, mostly clicks on the API help page example query, and a very few intentional queries.

select * from event.mediawiki_api_request
where
    params['action']='query' and params['prop']='references'
    and year=2019 and month=11
    limit 1000

Since the feature is disabled, the response is always an error citestoragedisabled.

According to this declined task T125329#5341754 and the (closed) task to remove the feature, T222373: Remove the lazy load references beta feature we are free to remove the API. I'll take the opportunity to do so.

I don't think we can remove this so easily. For example, why are there patches proposing changes to this API? https://gerrit.wikimedia.org/r/278702 Did we asked the user if he is using the API, or if he does have another reason to touch the code? As far as I know we can't reach him for now. But I suspect the API is in use somewhere. I feel this requires a longer deprecation phase – which is not something we can do as part of our current project.

I don't think we can remove this so easily. For example, why are there patches proposing changes to this API? https://gerrit.wikimedia.org/r/278702 Did we asked the user if he is using the API, or if he does have another reason to touch the code? As far as I know we can't reach him for now. But I suspect the API is in use somewhere. I feel this requires a longer deprecation phase – which is not something we can do as part of our current project.

Thanks for the response! The @Cenarium patch is from 2016 when the feature was still in active development. This developer was helping figure out the details of how the API might work. There's no evidence that the feature was ever used, even in third-party wikis. Happy to continue discussing the deprecation question, but I feel like we need at least one data point suggesting that anyone is relying on the functionality, and currently using MediaWiki >=1.34.

Change 552211 had a related patch set uploaded (by Awight; owner: Awight):
[mediawiki/extensions/Cite@master] Remove ApiQueryReferences support

https://gerrit.wikimedia.org/r/552211

(pulling into the sprint to help w/ discussion and review)

thiemowmde set the point value for this task to 1.Nov 26 2019, 12:04 PM
thiemowmde changed the point value for this task from 1 to 2.

Change 552211 merged by jenkins-bot:
[mediawiki/extensions/Cite@master] Remove ApiQueryReferences support

https://gerrit.wikimedia.org/r/552211

thiemowmde closed this task as Resolved.Dec 4 2019, 8:27 AM
thiemowmde claimed this task.
thiemowmde moved this task from Review to Done on the WMDE-QWERTY-Sprint-2019-11-20 board.

For reference: We announced the removal of this API on 2019-12-05 via wikitext-l. We expect it to disappear in the builds https://www.mediawiki.org/wiki/MediaWiki_1.35/wmf.10 and later in MediaWiki 1.36.

Change 615681 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/Cite@master] Remove unused Cite::$isPagePreview flag

https://gerrit.wikimedia.org/r/615681

Change 615681 merged by jenkins-bot:
[mediawiki/extensions/Cite@master] Remove unused Cite::$isPagePreview flag

https://gerrit.wikimedia.org/r/615681