Page MenuHomePhabricator

Allow segmentation for large WANCache values
Closed, ResolvedPublic

Description

While BagOStuff supports the storing of large values (>1mb) in memcached using WRITE_ALLOW_SEGMENTS, the WANObjectCache class does not support it yet. Since the preprocessor text was changed to use WANCache, large wiki pages will not benefit from caching since the cache writes will fail. The preprocessor cache is useful for expanding templates during parser cache misses (e.g. on edit or for users with non-standard preferences).

An example of large parser output entries was found during unrelated debug log entries for RefreshLinksJob failures at T307133.

We should:

  • Add a "segmentable" option to WANCache that uses WRITE_ALLOW_SEGMENTS
  • Use the new "segmentable" option in SqlBlobStore (which uses WANCache already)
  • Make PreprocessorHash use JSON again (like before de6eeead21f9cdb1bffa3f1b5877f71d03529722)
  • Use the new "segmentable" option in PreprocessorHash

I also noticed that PageEditStash can no longer segment values (since 91043c8fe8521216be365898eb7f5f06c495dd00), I'll spin that off as another task.

Event Timeline

Change 825410 had a related patch set uploaded (by Aaron Schulz; author: Aaron Schulz):

[mediawiki/core@master] objectcache: add "segmentable" option to WANObjectCache

https://gerrit.wikimedia.org/r/825410

Krinkle triaged this task as Medium priority.Nov 4 2022, 12:20 AM

Change 825410 merged by jenkins-bot:

[mediawiki/core@master] objectcache: add "segmentable" option to WANObjectCache

https://gerrit.wikimedia.org/r/825410

Change 855094 had a related patch set uploaded (by Aaron Schulz; author: Aaron Schulz):

[mediawiki/core@master] blobstore: rename confusing $blobKey parameters

https://gerrit.wikimedia.org/r/855094

Change 855095 had a related patch set uploaded (by Aaron Schulz; author: Aaron Schulz):

[mediawiki/core@master] blobstore: enable cache value segmentation for blobs

https://gerrit.wikimedia.org/r/855095

I also noticed that the PageEditStash values cannot use segmentation (since 91043c8fe852121) due to using object.

Change 855684 had a related patch set uploaded (by Aaron Schulz; author: Aaron Schulz):

[mediawiki/core@master] parser: allow segmentation of large preprocessor cache values

https://gerrit.wikimedia.org/r/855684

Change 855094 merged by jenkins-bot:

[mediawiki/core@master] blobstore: rename confusing $blobKey parameters

https://gerrit.wikimedia.org/r/855094

Change 855684 merged by jenkins-bot:

[mediawiki/core@master] parser: allow segmentation of large preprocessor cache values

https://gerrit.wikimedia.org/r/855684

Change 855095 merged by jenkins-bot:

[mediawiki/core@master] blobstore: enable cache value segmentation for blobs

https://gerrit.wikimedia.org/r/855095

aaron updated the task description. (Show Details)