Description
The orchestrator performs two very expensive operations on ZObejcts: eager evaluation and validation. These operations recurse over the entire object and may kick off other expensive operations, resulting in exponential-time performance degradation.
These operations should be run at most once over a given object (unless/until it experiences a state change that invalidates prior operations).
Moreover, sometimes these operations should not be run at all. Many objects, such as instances of Wikidata types, are constructed automatically by built-in functions in the orchestrator, which check the validity of the JSON content retrieved from Wikidata. it's inessential for the orchestrator to to validate these instances.
It should be possible to eliminate these steps by marking the ZWrappers for these instances as "already validated and checking the flag before running the validation step.
Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)
- We expect this to improve performance whenever the orchestrator fetches Wikidata content of a nontrivial size, and further reduce the occurrences of timeouts described in T377338.
- Memory usage should decrease by ~30% when run locally
- Duration on certain calls (like Echo( Dereference( LID ) )) should decrease by ~70% when run locally
- Validation is run at most once on objects (not counting re-runs due to state changes).
- Eager evaluation is run at most once on objects (not counting re-runs due to state changes).
- Validation and eager evaluation are never explicitly run on objects produced directly by the orchestrator, since these objects are known to be valid and evaluated.
Completion checklist
- Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist