Is this something Product want? If so, how should it work? Does this need a new UX? Should we encourage users to use this in replacement for the standard fetch Functions etc.?

Hi @Jdforrester-WMF - regarding the questions above –

• this hasn't been discussed with Product, but I will take an action item to make them aware of it.
• This would not involve any UX changes or any new functions or function arguments.
• This is a way to alleviate performance issues that we have with orchestrator processing of large Wikidata entities, simply by importing fewer statements. We would ignore external-ID statements when fetching a Wikidata entity.
• This would be very easy to arrange with a bit of feature-flag coding in the orchestrator, and the feature-flag could later be used to re-enable the import of these statements if and when they are ever really needed.
• Even though these are ordinary statements, they are placed in a separate section, with heading Identifiers, on Wikidata pages – I presume because they are considered to be of less interest than the other statements; less browse-worthy. Looking at the page for Chicago, I currently see 148 statements in this category (example: National Library of Brazil ID for the topic Chicago)

In T405991#11235278, @DMartin-WMF wrote:

Hi @Jdforrester-WMF - regarding the questions above –

• this hasn't been discussed with Product, but I will take an action item to make them aware of it.

No, David, that's not quite our process; you don't need to take an action. In the task triage yesterday it was agreed that Product needed to review this before any work was done on it, and I wrote down the questions we came up with in the meeting that need answering from them.

No further action should be done on this task until that happens.

No further action should be done on this task until that happens.

Okay, understood. The following relevant info may also be helpful:

• T405849 is a recent example of the orchestrator performance concerns that this would help with.
• The proposed work would only take about an hour of coding time.

I'm not keen on this unless specifically requested as a flag by the function call to Fetch. IMHO the identifiers are actually the most important part of a Wikidata item. They make it a universal identifier hub. In fact I wrote an entire app, Entity Explosion, based on this (T253201). I'm not sure that aspect will be used very much on Wikifunctions, but it could. For example, we could create an HTML fragment like https://en.wikipedia.org/wiki/Template:Taxonbar for an arbitrary item type.

Thanks, @99of9 . I agree this info may get some use in Wikifunctions, sooner or later. The idea behind this proposal is to non-permanently remove the external-ID statements, to alleviate serious performance issues that we are grappling with, until we've addressed those performance issues by more fundamental measures. Then we would reenable the inclusion of the external-ID statements.

I think it's a sensible trade-off if we can unbreak existing functionality, such as that described by T405849, by non-permanently turning off some content that's unlikely to be used for a while.

Also, as noted in the Description, when T382921 is done, there could still be a way for users to fetch the external-ID statements.

I don’t understand the delay on T382921. I don’t know the statistics, but I would expect labels and aliases in irrelevant languages would generally be bigger than external IDs, and the default omission of other languages would appear to be “common sense” to most users, I imagine. (Chicago has 224 labels, for example.)

I agree that T382921 is a more powerful/general solution to memory challenges, and I hope to move forward on it very soon. Note however that it places a greater burden on users - to get its benefits users will need to explicitly specify values for the planned optimization parameters (or create wrapper functions that encode those values). That's one reason why it hasn't made it to the top of the priority list, so far.

In T405991#11247620, @DMartin-WMF wrote:

… it places a greater burden on users …

We have broad shoulders! I would expect wrappers to do the heavy lifting, and some of these already exist. filter statements by property type (Z28548) and label strings from Wikidata item for language-list (Z24114) spring to mind.

It depends which users you have in mind, of course, but I paused my own work on re-ordered lexemes because it would have been necessary to maintain large chunks of duplicated code across multiple implementations, given that the performance at the time precluded chaining filters and sorts. More recent work on filtered statement lists is looking promising, but avoiding the initial overhead of handling the whole item, including contextually irrelevant labels, would be a significant step forward.

Not complaining here, but trust me: this should always have been a higher priority!

Thanks, @GrounderUK , for providing that context!

Re: Update, 26 May 2026
Yes, a general “negative filter” would be helpful, allowing the omission of population and GDP statistics from country items, for example. I have it planned as part of a broader post-fetch statement-filter approach and there my tentative conclusion was that each filter should be either positive or negative across all dimensions, just for usability. Otherwise, we risk building a query language. For the limited dimensions of a fetch like Z6820, however, I think we might manage a usable solution that allows each dimension to be specified either positively or negatively. I don’t see any real use case for language exclusions, but I imagine that most fetches should actually exclude either labels or statements. Similarly, I see no case for excluding qualifiers, whereas excluding references would be the norm. That said, if we bring statement identifiers into scope, I can envisage cases where we would initially filter reduced statements and then use the identifiers to fetch the expanded statements. For example, abstract:Q762 asserts that Q215486 is a notable work by Leonardo da Vinci, so we are first interested in the existence of Q762•P800•Q215486 (undeprecated), which happens to have no references, and then in Q215486•P170•Q762 which has two references, which happen to be project imports (P143) and therefore not, in themselves, legitimate references. (So the hypothetical function-call result is a call-to-action: add-reference-to-identified-statement.)

I want an opt-in filter, not an opt-out one. Items like Q183 (Germany) is still quite large even if external IDs are stripped. If I want only one label, then fetch and return that label only, not fetching anything else.

In T405991#12125384, @Bugreporter wrote:

I want an opt-in filter, not an opt-out one. Items like Q183 (Germany) is still quite large even if external IDs are stripped. If I want only one label, then fetch and return that label only, not fetching anything else.

Yes, that's an important need. Z6820 / Fetch Wikidata entities is meant to meet that need, and various wrapper-functions that call it are also possible, To provide greater ease-of-use, such as fetch Wikidata item or parts (Z30120).

The suggested "opt-in" by listing PIDs was added some time ago.
To skip external IDs, filter with Z35126 before fetching, or with Z29724 afterwards. You'll notice some mistakes since the hardcoded list it draws from was only populated with the top ~80 by usage, not exhaustive and not up-to-date.

Thanks, @YoshiRulz ! I am very glad to know about this.