==== Error ====
* service.version: 1.41.0-wmf.12
* trace.id: c5dcc0ea-bf98-4040-9172-9c4494145ed1
* [[ https://logstash.wikimedia.org/app/dashboards#/view/AXFV7JE83bOlOASGccsT?_g=(time:(from:'2023-06-12T07:13:26.930Z',to:'2023-06-13T08:57:21.152Z'))&_a=(query:(query_string:(query:'reqId:%22c5dcc0ea-bf98-4040-9172-9c4494145ed1%22'))) | Find trace.id in Logstash ]]
```name=labels.normalized_message,lines=10
[{reqId}] {exception_url} MediaWiki\Revision\BadRevisionException: The content of this revision is missing or corrupted (bad schema)
```
```name=error.stack_trace,lines=10
from /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionStore.php(1190)
#0 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionStore.php(1506): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, NULL, NULL, NULL, integer)
#1 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#2 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/SlotRecord.php(324): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#3 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionRecord.php(183): MediaWiki\Revision\SlotRecord->getContent()
#4 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/Config/PageContent.php(80): MediaWiki\Revision\RevisionRecord->getContentOrThrow(string)
#5 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Wt2Html/PageConfigFrame.php(27): MediaWiki\Parser\Parsoid\Config\PageContent->getContent(string)
#6 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Config/Env.php(285): Wikimedia\Parsoid\Wt2Html\PageConfigFrame->__construct(Wikimedia\Parsoid\Config\Env, MediaWiki\Parser\Parsoid\Config\PageConfig, MediaWiki\Parser\Parsoid\Config\SiteConfig)
#7 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Parsoid.php(162): Wikimedia\Parsoid\Config\Env->__construct(MediaWiki\Parser\Parsoid\Config\SiteConfig, MediaWiki\Parser\Parsoid\Config\PageConfig, MediaWiki\Parser\Parsoid\Config\DataAccess, ParserOutput, array)
#8 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Parsoid.php(216): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#9 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(298): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#10 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(465): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parseInternal(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#11 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(244): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parse(MediaWiki\Page\PageStoreRecord, ParserOptions, array, MediaWiki\Revision\RevisionStoreRecord)
#12 /srv/mediawiki/php-1.41.0-wmf.12/includes/jobqueue/jobs/ParsoidCachePrewarmJob.php(127): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->getParserOutput(MediaWiki\Page\PageStoreRecord, ParserOptions, MediaWiki\Revision\RevisionStoreRecord, integer)
#13 /srv/mediawiki/php-1.41.0-wmf.12/includes/jobqueue/jobs/ParsoidCachePrewarmJob.php(140): ParsoidCachePrewarmJob->doParsoidCacheUpdate()
#14 /srv/mediawiki/php-1.41.0-wmf.12/extensions/EventBus/includes/JobExecutor.php(79): ParsoidCachePrewarmJob->run()
#15 /srv/mediawiki/rpc/RunSingleJob.php(77): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#16 {main}
```
```name=error.stack.previous_trace,lines=10
from /srv/mediawiki/php-1.41.0-wmf.12/includes/Storage/SqlBlobStore.php(279)
#0 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionStore.php(1188): MediaWiki\Storage\SqlBlobStore->getBlob(string, integer)
#1 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionStore.php(1506): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, NULL, NULL, NULL, integer)
#2 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#3 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/SlotRecord.php(324): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#4 /srv/mediawiki/php-1.41.0-wmf.12/includes/Revision/RevisionRecord.php(183): MediaWiki\Revision\SlotRecord->getContent()
#5 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/Config/PageContent.php(80): MediaWiki\Revision\RevisionRecord->getContentOrThrow(string)
#6 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Wt2Html/PageConfigFrame.php(27): MediaWiki\Parser\Parsoid\Config\PageContent->getContent(string)
#7 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Config/Env.php(285): Wikimedia\Parsoid\Wt2Html\PageConfigFrame->__construct(Wikimedia\Parsoid\Config\Env, MediaWiki\Parser\Parsoid\Config\PageConfig, MediaWiki\Parser\Parsoid\Config\SiteConfig)
#8 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Parsoid.php(162): Wikimedia\Parsoid\Config\Env->__construct(MediaWiki\Parser\Parsoid\Config\SiteConfig, MediaWiki\Parser\Parsoid\Config\PageConfig, MediaWiki\Parser\Parsoid\Config\DataAccess, ParserOutput, array)
#9 /srv/mediawiki/php-1.41.0-wmf.12/vendor/wikimedia/parsoid/src/Parsoid.php(216): Wikimedia\Parsoid\Parsoid->parseWikitext(MediaWiki\Parser\Parsoid\Config\PageConfig, ParserOutput, array)
#10 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(298): Wikimedia\Parsoid\Parsoid->wikitext2html(MediaWiki\Parser\Parsoid\Config\PageConfig, array, NULL, ParserOutput)
#11 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(465): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parseInternal(MediaWiki\Parser\Parsoid\Config\PageConfig, array)
#12 /srv/mediawiki/php-1.41.0-wmf.12/includes/parser/Parsoid/ParsoidOutputAccess.php(244): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->parse(MediaWiki\Page\PageStoreRecord, ParserOptions, array, MediaWiki\Revision\RevisionStoreRecord)
#13 /srv/mediawiki/php-1.41.0-wmf.12/includes/jobqueue/jobs/ParsoidCachePrewarmJob.php(127): MediaWiki\Parser\Parsoid\ParsoidOutputAccess->getParserOutput(MediaWiki\Page\PageStoreRecord, ParserOptions, MediaWiki\Revision\RevisionStoreRecord, integer)
#14 /srv/mediawiki/php-1.41.0-wmf.12/includes/jobqueue/jobs/ParsoidCachePrewarmJob.php(140): ParsoidCachePrewarmJob->doParsoidCacheUpdate()
#15 /srv/mediawiki/php-1.41.0-wmf.12/extensions/EventBus/includes/JobExecutor.php(79): ParsoidCachePrewarmJob->run()
#16 /srv/mediawiki/rpc/RunSingleJob.php(77): MediaWiki\Extension\EventBus\JobExecutor->execute(array)
#17 {main}
```
==== Notes ====
Parsoid cache prewarm job started showing this problem shortly after deployment of 1.41.0-wmf.12 to group2 on Thursday last week (June 8th).
It's unclear where the problem originates exactly; neither the parsoid code nor the revision store code had code modifications in the time relevant time frame, so I'm tagging the ticket with both components. Could be a data corruption problem, but then it's strange that the problem started happening right after the code deployment.