Page MenuHomePhabricator

Wikidata import: always create values for Z6003K5/qualifiers and Z6003K6/references
Closed, ResolvedPublic

Description

Description

In the initial implementation of orchestrator code to import qualifiers and references, we decided to omit Z6003K5 or Z6003K6 if there were no qualifiers or references, respectively. This was solely motivated by the desire to produce more compact ZObjects (for imported Wikidata entities), and thus reduce (somewhat) the amount of processing the orchestrator does over such objects. (There can be many statements inside a Wikidata entity, and many of them may have no qualifiers and/or references. The normal form representation of an empty list of ZObjects is a bit bulky, so we wanted to omit those empty lists.)

This turned out to be a mistake. When an imported statement has a missing Z6003K5 or Z6003K6, it cannot be compared to a statement constructed in the UI, where those keys will always appear (Without making an unnecessarily complex comparison function). So we need to reverse that decision.

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • Every statement in an imported Wikidata entity should have a value for Z6003K5 and Z6003K6. When there are no qualifiers or references, the value should be an empty Typed List with the appropriate type.

Completion checklist

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Always create values for Z6003K5/qualifiers and Z6003K6/referencesrepos/abstract-wiki/wikifunctions/function-orchestrator!502dmartinT411947main
Customize query in GitLab

Event Timeline

DMartin-WMF renamed this task from Wikidata import: create a value for Z6003K5/qualifiers and Z6003K6/references to Wikidata import: always create values for Z6003K5/qualifiers and Z6003K6/references.Dec 7 2025, 2:00 AM
DMartin-WMF triaged this task as Medium priority.
DMartin-WMF updated the task description. (Show Details)
DMartin-WMF changed the task status from Open to In Progress.Dec 15 2025, 1:01 AM
DMartin-WMF claimed this task.

Change #1224079 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-12-10-133418 to 2026-01-07-132737

https://gerrit.wikimedia.org/r/1224079

Change #1224079 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-12-10-133418 to 2026-01-07-132737

https://gerrit.wikimedia.org/r/1224079

Jdforrester-WMF subscribed.

@DMartin-WMF: This has now been deployed; please validate and mark as Resolved, or move back to In Engineering, as appropriate.

As expected, some test cases were failing when revisited in edit mode and passing when an empty list replaced the call to nullary void (Z26199) for the arguments to emulate Wikidata statement object (Z23723).

I’ve now changed the only implementation for Z23723, emulate Wikidata statement object, Python (Z28538), but an empty list is typed as Z1 (unavoidably, I believe), so some tests will still fail. One example is NSW timezones (Z29836). I think these can be fixed by replacing the nullary void Function calls with literal empty lists, which is a lot simpler than re-building the entire test. @99of9

There are only 13 tests using this function. Do we want to add these to Request for cleanup? I don’t see any active harm in leaving them to call the emulator; converting to a statement constructed in the UI is an alternative, but that’s a non-trivial task.

I don't mind changing it, but it appears to be passing all implementations at the moment?

I don't mind changing it, but it appears to be passing all implementations at the moment?

Ah, good. Presumably that was T406784 biting again.

Test case Z30130 appears to fail with implementation Z30129, however. I’m not sure what the issue is there, but it appears to be something to do with using Z6803 as the element equality function in Z889, and T407700#11501686 does not seem to apply here, since the Z6003K5 and Z6003K6 are actually empty, as expected, and the test case passes with implementation Z30127.

I have checked the other test cases using statement emulation, and these all now seem okay (ignoring “defining qualities of Wikidata item (Z29865), which has no connected implementation). I’ll probably re-implement Z23723 in due course, completely removing the option for omitting keys (which is currently overridden only for Z6003K5 and Z6003K6); I don’t see this as a priority, however.