Page MenuHomePhabricator

[Stretch] Update orchestrator code of Wikidata Fetch functions to instantiate references
Closed, ResolvedPublic

Description

Description

The relevant code is in transform.js. References will be nested inside of statements, which are already imported. Since all of the entity types contain statements, all of the entity fetch functions should be able to import references as a result of this task.

  • Create new instances of Z6008/Wikidata reference when fetch functions run.
    • This could be done by a new function convertReferences, somewhat similar to convertQualifiers.
    • convertReferences should be able to make use of convertSnak.
    • The "hash" and "snaks-order" fields (in the Wikidata JSON) can be ignored. However, the ordering of snaks there should be maintained in each reference's Z6008K1 property.
  • List those instances in the new property Z6003K6.
    • To do this, convertReferences would be called inside convertStatement, after the place where convertQualifiers is called
  • Update any affected existing tests' *_ expected.json files with the newly generated reference ZObjects.

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • Handle all qualifiers that are used in Wikidata, as they are presented in Wikidata's JSON files returned from the relevant APIs.

Remove all the non-applicable tags from the "Tags" field, leave only the tags of the projects/repositories related to this task


Completion checklist

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Wikidata Fetch functions instantiate referencesrepos/abstract-wiki/wikifunctions/function-orchestrator!428ecarggrace/T399920/wikidata-refs-instantiationmain
Draft: Wikidata Fetch functions instantiate referencesrepos/abstract-wiki/wikifunctions/function-orchestrator!427ecarggrace/T399920/import-refsmain
Customize query in GitLab

Event Timeline

DSantamaria added a project: OKR-Work.
ecarg changed the task status from Open to In Progress.Aug 14 2025, 9:14 PM
ecarg claimed this task.
DSantamaria changed the task status from In Progress to Open.Aug 22 2025, 2:16 PM

This is merged, but not yet switched on. Do we want to keep this task for that part of the work (in which case, this should go back to In Engineering), or do that in a different task (in which case this should go to Waiting for Deployment)?

I'm thinking a different task makes more sense, since we currently don't know how long it'll be before we choose to switch this on. I will make a different task. Also, I'm thinking the new task doesn't need to be part of the Q1 epic, because references is labeled as a stretch goal and we don't have any success metrics related to the references.

Change #1189156 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-09-08-191243 to 2025-09-16-134119

https://gerrit.wikimedia.org/r/1189156

Change #1189156 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2025-09-08-191243 to 2025-09-16-134119

https://gerrit.wikimedia.org/r/1189156