Instead of JSON consider a serialization that supports random access and possibly can even be used for in memory representation for storage in the DB. This would make the work for deferred deserialization even less.
Something like BJSON, Protocol Buffers or EXI, where you can know the length of something without searching.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Addshore | T90706 [Task] Profile entity deserialization | |||
| Open | None | T90707 [Task] Implement deferred deserialization of Entity | |||
| Open | None | T88728 Improve Wikimedia dumping infrastructure | |||
| Open | None | T88991 improve Wikidata dumps [tracking] | |||
| Open | None | T119612 Consider a serialization that supports random access for storage in the DB for Wikidata |
This would still be great, but it is not high prio.
Tagging SRE given the discussions we have a 2018 Tech Conf about storage types.