When exporting articles from a Wikipedia (through [[Special:Export]]), or in the XML dump, there is no mention of the Wikidata qid corresponding to each article.
It would be great to have it added, to save further API queries to get it or to avoid having to download a full dump of Wikidata on top of the Wikipedia(s) one.
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| Add wikibase QID to XML export/dump | Suzanne Wood | mediawiki/extensions/Wikibase | master | +90 -1 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | None | T197090 [CLIENT][SW] Wikidata qid of articles is not present in export/dump | |||
| Resolved | None | T379615 Exporting articles from a Wikipedia (through [[Special:Export]]) | |||
| Resolved | None | T379617 Exporting articles from a Wikipedia in the XML dump |
I suppose the WikibaseClient would need to attach to one of the hooks in XmlDumpWriter or something similar, like XmlDumpWriterOpenPage.
I worry that this will make the dumps that much slower. Thre ought to be some sort of entity dump for Wikidata that could be used in tandem with the regular dumps. Adding @hoo for comments.
This page about the wikibase dump format says that we follow the Stable Interface Policy, which mentions about XML specifically that “For data formats that allow namespacing, like XML does, names (attribute names, element names) that belong to a namespace not explicitly mentioned by the specification of the data format can be ignored by consumers. Addition and changes to data structures from other namespaces are not considered breaking changes.”
Hi @Sylvain_WMFr, my team is investigating this ticket. We're curious about what types of editors create XML exports. Could you tell us what you would require the Wikidata QID for in your exports? Thank you.
Size of impact on big dump:
I realise my estimate before was wrong of 100 million rows, it will actually be around 7 million rows added
(I was looking at wikidata items before but not all of them have a corresponding site-linked wikipedia page)
Hi! (answering from my personal account as I don't work at Wikimédia France anymore since 2020)
Sorry, I don't remember the details as this ticket is so old... I guess this had to do with the Ma Commune project, which is now offline, so I don't think there is still a need for it.
Change #1100115 had a related patch set uploaded (by Suzanne Wood; author: Suzanne Wood):
[mediawiki/extensions/Wikibase@master] Add wikibase QID to XML export/dump
Current status of this ticket:
We are currently not planning to deploy this since:
However, should a need arise in the future, a commit is ready here
Additional context:
We’ve contacted the WMF and they said that XML exports are on maintenance only mode and are being replaced by the new event-driven Dumps 2.0 system - we have mentioned considering incorporating QIDs here. It may also be worth coordinating with this ticket which mentioned potential XML dump changes: T365693
@SuzanneWood-WMDE thanks for the update, please reach out to the DPE / myself regarding anything Dumps 2 (and 1) related going forward.