Change Details

We know that Kartographer stores information about maps on a page in ParserCache extension data, and this data can change with each revision of a page. We want to know what happens to old revisions of the data, especially the stable page revision when using FlaggedRevs. Is the stable revision cached separately from the head revision? Is the head revision also cached? If they are both cached, why does it look like both share the same cache key? Is stable revision content kept in a normal LRU along with the rest of the cached content? == Main findings == * FlaggedRevs maintains an entirely separate ParserCache for stable revision content, named `stable-pcache`. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#ParserCache | (1) ]] [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/FlaggedRevs/+/refs/heads/master/frontend/FlaggablePageView.php#681 | (2) ]] == Other findings == * ParserCache is configured to expire after 21 days. [[ https://wikitech.wikimedia.org/wiki/Parser_cache | (3) ]] * If we need to lower the retention period for a given page, it can be done by calling `ParserOutput->updateCacheExpiry()`. * In production, ParserCache is not backed by a standard cache, but is a custom SQL table with its own incredibly expensive manual purge mechanism. [[ https://phabricator.wikimedia.org/T282761 | (4) ]] We migrated from memcache to sql backing in 2011. There may be an additional memcache tier "on-host" (not sure what this means) in some cases. [[ https://phabricator.wikimedia.org/T264604 | (5) ]] * One consequence of (4) is that we need to be careful not to increase the amount of data in ParserCache. * Entries in the main ParserCache are also invalidated when a page's `page.page_touched` timestamp is updated. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#Population,_Invalidation,_Expiry,_and_Eviction | (6) ]] * Hit rates are stable at c. 80%. [[ https://grafana.wikimedia.org/d/000000106/parser-cache?orgId=1 | (7) ]] * There's a separate RevisionOutputCache which seems to have the same content as ParserCache, but is meant for storing output for specific revisions in case one revision is being accessed repeatedly (for example, an oldid link is published on the Internet). I'm not sure if this is exactly the same content or if the format differs, and whether it can be accessed transparently as if it were part of ParserCache. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#RevisionOutputCache | (8) ]].

We know that Kartographer stores information about maps on a page in ParserCache extension data, and this data can change with each revision of a page. We want to know what happens to old revisions of the data, especially the stable page revision when using FlaggedRevs. Is the stable revision cached separately from the head revision? Is the head revision also cached? If they are both cached, why does it look like both share the same cache key? Is stable revision content kept in a normal LRU along with the rest of the cached content? == Main findings == * FlaggedRevs maintains an entirely separate ParserCache for stable revision content, named `stable-pcache`. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#ParserCache | (1) ]] [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/FlaggedRevs/+/refs/heads/master/frontend/FlaggablePageView.php#681 | (2) ]] * ParserCache key does not include revision ID. == Other findings == * ParserCache is configured to expire after 21 days. [[ https://wikitech.wikimedia.org/wiki/Parser_cache | (3) ]] * If we need to lower the retention period for a given page, it can be done by calling `ParserOutput->updateCacheExpiry()`. * In production, ParserCache is not backed by a standard cache, but is a custom SQL table with its own incredibly expensive manual purge mechanism. [[ https://phabricator.wikimedia.org/T282761 | (4) ]] We migrated from memcache to sql backing in 2011. There may be an additional memcache tier "on-host" (not sure what this means) in some cases. [[ https://phabricator.wikimedia.org/T264604 | (5) ]] * One consequence of (4) is that we need to be careful not to increase the amount of data in ParserCache. * Entries in the main ParserCache are also invalidated when a page's `page.page_touched` timestamp is updated. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#Population,_Invalidation,_Expiry,_and_Eviction | (6) ]] * Hit rates are stable at c. 80%. [[ https://grafana.wikimedia.org/d/000000106/parser-cache?orgId=1 | (7) ]] * There's a separate RevisionOutputCache which seems to have the same content as ParserCache, but is meant for storing output for specific revisions in case one revision is being accessed repeatedly (for example, an oldid link is published on the Internet). I'm not sure if this is exactly the same content or if the format differs, and whether it can be accessed transparently as if it were part of ParserCache. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#RevisionOutputCache | (8) ]].

We know that Kartographer stores information about maps on a page in ParserCache extension data, and this data can change with each revision of a page. We want to know what happens to old revisions of the data, especially the stable page revision when using FlaggedRevs. Is the stable revision cached separately from the head revision? Is the head revision also cached? If they are both cached, why does it look like both share the same cache key? Is stable revision content kept in a normal LRU along with the rest of the cached content? == Main findings == * FlaggedRevs maintains an entirely separate ParserCache for stable revision content, named `stable-pcache`. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#ParserCache | (1) ]] [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/extensions/FlaggedRevs/+/refs/heads/master/frontend/FlaggablePageView.php#681 | (2) ]] * ParserCache key does not include revision ID. == Other findings == * ParserCache is configured to expire after 21 days. [[ https://wikitech.wikimedia.org/wiki/Parser_cache | (3) ]] * If we need to lower the retention period for a given page, it can be done by calling `ParserOutput->updateCacheExpiry()`. * In production, ParserCache is not backed by a standard cache, but is a custom SQL table with its own incredibly expensive manual purge mechanism. [[ https://phabricator.wikimedia.org/T282761 | (4) ]] We migrated from memcache to sql backing in 2011. There may be an additional memcache tier "on-host" (not sure what this means) in some cases. [[ https://phabricator.wikimedia.org/T264604 | (5) ]] * One consequence of (4) is that we need to be careful not to increase the amount of data in ParserCache. * Entries in the main ParserCache are also invalidated when a page's `page.page_touched` timestamp is updated. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#Population,_Invalidation,_Expiry,_and_Eviction | (6) ]] * Hit rates are stable at c. 80%. [[ https://grafana.wikimedia.org/d/000000106/parser-cache?orgId=1 | (7) ]] * There's a separate RevisionOutputCache which seems to have the same content as ParserCache, but is meant for storing output for specific revisions in case one revision is being accessed repeatedly (for example, an oldid link is published on the Internet). I'm not sure if this is exactly the same content or if the format differs, and whether it can be accessed transparently as if it were part of ParserCache. [[ https://www.mediawiki.org/wiki/Manual:Parser_cache#RevisionOutputCache | (8) ]].