Possibly a duplicate of T351953, but this is a different wiki so I figured I might as well file a new task.

After checking several things to try to find more examples without success (including finally checking the diffs), I've come to the conclusion that these were gzipped revisions affected by the now-resolved T21990 from 2009. This also occurs on a page mentioned in that bug report (for example: https://en.wikipedia.org/w/index.php?title=Clearwater_River_(Idaho)&oldid=304883907). Once I'd figured out the nature of the bug, I stopped undeleting these revisions until it was fixed.

For reference, the exception is:

PCRE failure
from /srv/mediawiki/php-1.44.0-wmf.17/includes/parser/Parser.php(2172)
#0 /srv/mediawiki/php-1.44.0-wmf.17/includes/parser/Parser.php(1632): MediaWiki\Parser\Parser->handleExternalLinks(string)
#1 /srv/mediawiki/php-1.44.0-wmf.17/includes/parser/Parser.php(701): MediaWiki\Parser\Parser->internalParse(string)
#2 /srv/mediawiki/php-1.44.0-wmf.17/includes/content/WikitextContentHandler.php(384): MediaWiki\Parser\Parser->parse(string, MediaWiki\Title\Title, MediaWiki\Parser\ParserOptions, bool, bool, int)
#3 /srv/mediawiki/php-1.44.0-wmf.17/includes/content/ContentHandler.php(1697): MediaWiki\Content\WikitextContentHandler->fillParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams, MediaWiki\Parser\ParserOutput)
#4 /srv/mediawiki/php-1.44.0-wmf.17/includes/content/Renderer/ContentRenderer.php(75): MediaWiki\Content\ContentHandler->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Content\Renderer\ContentParseParams)
#5 /srv/mediawiki/php-1.44.0-wmf.17/includes/Revision/RenderedRevision.php(261): MediaWiki\Content\Renderer\ContentRenderer->getParserOutput(MediaWiki\Content\WikitextContent, MediaWiki\Page\PageIdentityValue, MediaWiki\Revision\RevisionStoreRecord, MediaWiki\Parser\ParserOptions, array)
#6 /srv/mediawiki/php-1.44.0-wmf.17/includes/Revision/RenderedRevision.php(233): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(MediaWiki\Content\WikitextContent, array)
#7 /srv/mediawiki/php-1.44.0-wmf.17/includes/Revision/RevisionRenderer.php(236): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string, array)
#8 /srv/mediawiki/php-1.44.0-wmf.17/includes/Revision/RevisionRenderer.php(169): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, MediaWiki\Parser\ParserOptions, array)
#9 /srv/mediawiki/php-1.44.0-wmf.17/includes/Revision/RenderedRevision.php(196): MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#10 /srv/mediawiki/php-1.44.0-wmf.17/includes/poolcounter/PoolWorkArticleView.php(106): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#11 /srv/mediawiki/php-1.44.0-wmf.17/includes/poolcounter/PoolWorkArticleViewOld.php(81): MediaWiki\PoolCounter\PoolWorkArticleView->renderRevision(null, bool, string)
#12 /srv/mediawiki/php-1.44.0-wmf.17/includes/poolcounter/PoolCounterWork.php(171): MediaWiki\PoolCounter\PoolWorkArticleViewOld->doWork()
#13 /srv/mediawiki/php-1.44.0-wmf.17/includes/page/ParserOutputAccess.php(369): MediaWiki\PoolCounter\PoolCounterWork->execute()
#14 /srv/mediawiki/php-1.44.0-wmf.17/includes/page/Article.php(829): MediaWiki\Page\ParserOutputAccess->getParserOutput(WikiPage, MediaWiki\Parser\ParserOptions, MediaWiki\Revision\RevisionStoreRecord, int)
#15 /srv/mediawiki/php-1.44.0-wmf.17/includes/page/Article.php(545): Article->generateContentOutput(MediaWiki\User\User, MediaWiki\Parser\ParserOptions, int, MediaWiki\Output\OutputPage, array)

In the specific case of User:SoniC~enwiki, I've re-imported the corrupted edits from "SoniC" in the May 2003 database dump, taken the opportunity to import another edit from the Nostalgia Wikipedia, and moved the corrupted edits to: https://en.wikipedia.org/wiki/User:SoniC~enwiki/Duplicate_corrupted_edits

(For the curious - it really is just gzipped with no other mangling, so you can download the raw bytes from https://en.wikipedia.org/w/index.php?title=Clearwater_River_(Idaho)&oldid=304883907&action=raw, run them through the php gzinflate function, and get text back)

Edit later: the content is gzipped but in a weird format which makes most non-PHP gzip tools barf. Python can accept it only if you use a decompressobj with wbits=-zlib.MAX_WBITS, otherwise it complains about an incomplete or truncated stream.

And there's also a few instances like https://en.wikipedia.org/w/index.php?title=Clearwater_River_(Idaho)&oldid=304883903 where the content is DB://cluster1/1207116, as another manifestation of the same issue. In that case the data requires a sysadmin to fix or figure out what's there, but shouldn't be totally lost.

(Although in the specific case of Clearwater River it looks like the content should also be available in https://dumps.wikimedia.org/archive/2005/2005-03-16/)

I wrote a quick script to find other revisions that are likely affected by the same bug (have revids around the time, and old timestamps). It found:

296562614 0 Coconut_crab undecodable
297160170 1 Eritrea undecodable
297160171 1 Eritrea undecodable
297666320 0 Yogurt undecodable
298956368 0 Sámi_languages undecodable
300054141 0 Rope_(film) undecodable
300054142 0 Rope_(film) undecodable
300952651 0 Chibiusa undecodable
300952652 0 Chibiusa undecodable
302545854 0 Debbie_Rowe undecodable
302545855 0 Debbie_Rowe undecodable
302545856 0 Debbie_Rowe undecodable
302545857 0 Debbie_Rowe undecodable
302545858 0 Debbie_Rowe undecodable
302545859 0 Debbie_Rowe undecodable
302545860 0 Debbie_Rowe undecodable
302545861 0 Debbie_Rowe undecodable
302545862 0 Debbie_Rowe undecodable
302545863 0 Debbie_Rowe undecodable
302545864 0 Debbie_Rowe undecodable
302545865 0 Debbie_Rowe undecodable
302545866 0 Debbie_Rowe undecodable
302545867 0 Debbie_Rowe undecodable
302545868 0 Debbie_Rowe undecodable
302545869 0 Debbie_Rowe undecodable
302956816 0 Lifeless_(EP) undecodable
302956817 0 Lifeless_(EP) undecodable
302956818 0 Lifeless_(EP) undecodable
303160834 2 SoniC~enwiki/Duplicate_corrupted_edits undecodable
303160835 2 SoniC~enwiki/Duplicate_corrupted_edits undecodable
303160836 2 SoniC~enwiki/Duplicate_corrupted_edits undecodable
303321551 0 Renee_Hartevelt undecodable
303497031 2 Amillar/Contributions undecodable
303497032 2 Amillar/Contributions undecodable
303497033 2 Amillar/Contributions undecodable
303497034 2 Amillar/Contributions undecodable
303497585 2 Amillar/Pictures undecodable
303497586 2 Amillar/Pictures undecodable
303865391 0 History_of_the_U.S._Virgin_Islands undecodable
303865392 0 History_of_the_U.S._Virgin_Islands undecodable
303865393 0 History_of_the_U.S._Virgin_Islands undecodable
303865394 0 History_of_the_U.S._Virgin_Islands undecodable
304117010 2 Xiemaisi undecodable
304117011 2 Xiemaisi undecodable
304117012 2 Xiemaisi undecodable
304117013 2 Xiemaisi undecodable
304117014 2 Xiemaisi undecodable
304883903 0 Clearwater_River_(Idaho)  looks like URL
304883904 0 Clearwater_River_(Idaho) undecodable
304883905 0 Clearwater_River_(Idaho) undecodable
304883906 0 Clearwater_River_(Idaho) undecodable
304883907 0 Clearwater_River_(Idaho) undecodable
304883908 0 Clearwater_River_(Idaho) undecodable
304883909 0 Clearwater_River_(Idaho) undecodable
304883910 0 Clearwater_River_(Idaho) undecodable
304883911 0 Clearwater_River_(Idaho) undecodable
304883912 0 Clearwater_River_(Idaho) undecodable
304883913 0 Clearwater_River_(Idaho) undecodable
304883914 0 Clearwater_River_(Idaho) undecodable
304883915 0 Clearwater_River_(Idaho)  looks like URL
304883916 0 Clearwater_River_(Idaho) undecodable
304883917 0 Clearwater_River_(Idaho) undecodable
304884769 0 Michael_Collins undecodable
304884770 0 Michael_Collins undecodable
304884771 0 Michael_Collins undecodable
304884772 0 Michael_Collins undecodable
304884773 0 Michael_Collins undecodable
304884774 0 Michael_Collins undecodable
305571806 0 Woo_woo undecodable
305571808 0 Woo_woo undecodable
306304318 0 Peter_Doherty_(immunologist) undecodable
307333412 2 Jornhj~enwiki undecodable
307333413 2 Jornhj~enwiki undecodable
307363340 0 Gefjon undecodable
307363341 0 Gefjon undecodable
307363342 0 Gefjon undecodable
307363343 0 Gefjon undecodable
307363344 0 Gefjon undecodable
307363345 0 Gefjon undecodable
307363346 0 Gefjon undecodable
307363347 0 Gefjon undecodable
307363348 0 Gefjon undecodable
309273963 5 Historical_archive/Logs/Deletion_log undecodable
309553727 11 Jazz/version_1 undecodable

Wow. Some of these were clearly accidental undeletions which should probably be reversed (by the history merge special page or something), like the coconut crab example, which would've been a redirect:
https://en.wikipedia.org/w/index.php?title=Coconut_crab&diff=prev&oldid=296562614

I'm wondering whether, as the only non-sysadmin able to do this (being a non-admin importer with working old database dumps), I should try to use database imports to fix the pages I can, particularly the 2001 user pages that might have other revisions floating around somewhere. I know this is relatively hacky but might be a reasonable solution in the meantime. If I do this I'll report them all at once here. I think I'd prioritise the 2001 edits and the accidental undeletions.

Of those:

298956368 0 Sámi_languages undecodable
307333412 2 Jornhj~enwiki undecodable
307333413 2 Jornhj~enwiki undecodable
307363340 0 Gefjon undecodable
307363341 0 Gefjon undecodable
307363342 0 Gefjon undecodable
307363343 0 Gefjon undecodable
307363345 0 Gefjon undecodable
307363346 0 Gefjon undecodable
307363347 0 Gefjon undecodable
307363348 0 Gefjon undecodable

Are actually a different bug - the content (recoverable with action=raw) is otherwise valid text in some non-utf8 encoding, not gzipped. I suspect the root cause may be the same, though - they got missed in some legacyEncoding fix due to the deletion/undeletion.

Oddly https://en.wikipedia.org/wiki/?oldid=307363341 and https://en.wikipedia.org/wiki/?oldid=307363340 produce a different exception than the others

[5eb12627-2977-4b4c-aa79-8f6c2bcefdbd] 2025-02-26 03:51:14: Fatal exception of type "TypeError"

Oh I was going to mention: these wouldn't have caused runtime errors until the resolution of T128151 in June 2023. There was also another bug relating to revisions not appearing properly, T22757, around mid-2009. This happened to be one of my peak periods for page history investigation ...

In T387188#10581446, @Graham87 wrote:

Wow. Some of these were clearly accidental undeletions which should probably be reversed (by the history merge special page or something), like the coconut crab example, which would've been a redirect:
https://en.wikipedia.org/w/index.php?title=Coconut_crab&diff=prev&oldid=296562614

It is. The full content of the revision (verified from action=raw and gzdeflate) is

#redirect [[Coconut Crab]]

(with a trailing newline)

I'm wondering whether, as the only non-sysadmin able to do this (being a non-admin importer with working old database dumps), I should try to use database imports to fix the pages I can, particularly the 2001 user pages that might have other revisions floating around somewhere. I know this is relatively hacky but might be a reasonable solution in the meantime. If I do this I'll report them all at once here. I think I'd prioritise the 2001 edits and the accidental undeletions.

Obviously up to you, but seems in-principle reasonable. Although it might be best to let some admin like myself handle the accidental undeletions rather than foisting them onto the CSD queue in an ugly manner. And the data wasn't lost here - just mangled in a recoverable fashion.

I wonder whether I should run for importuploader on enwiki if I'm going to keep doing things like this. Maybe not since I have fringe views about using XML import to rebase edits (https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive362#Please_review_my_revdel), but on the other hand as an admin and interface admin I am already trusted to respect consensus and not do stupid things and I already have XML import rights on MediaWiki.org.

In T387188#10581448, @Pppery wrote:

Of those: [snip rev ID numbers and article names]
Are actually a different bug - the content (recoverable with action=raw) is otherwise valid text in some non-utf8 encoding, not gzipped. I suspect the root cause may be the same, though - they got missed in some legacyEncoding fix due to the deletion/undeletion.

They'd be encoded in Windows-1252, nominally ISO/IEC 8859-1, which Wikipedia used until being upgraded to MediaWiki 1.5 in June 2005. That same MediaWiki version changed the way revisions were stored (both deleted and undeleted), meaning that revisions deleted before the English Wikipedia was upgraded to MW 1.5 had out-of-order ID numbers and were also sometimes gzipped, depending on when they were deleted.

It looks like giving out daily updates might be wise, as these are taking longer to deal with than I expected. I've moved the undecodable edits (along with other redirect-related junk) out of the way to /temp pages at Coconut crab, Yogurt, Sámi languages (where I re-imported the only non-redirect edit), Rope (film), and Chibiusa. I've found it useful to check rev ID's just after the undecodable edits, as with the GZ examples, the latest edit in a set of deleted edits wasn't compressed. I only just noticed the advice about not foisting too much on the CSD queue in an ugly manner ... yeah, good point. I figured out that if I history-split the most recent edits first, I only need to use one temp page, thus minimising the damage that way. Also quick and dirty database queries like this are good for checking my work:
https://quarry.wmcloud.org/query/91184

Re importuploader: I'd support you for that, but hehehe re your link; I for one would've just left that case alone, with no revdels or anything. But rebasing is tempting sometimes; I wish there was a way to do it for the UseModWiki edits imported in September 2002, with their extra line breaks, while keeping their old revid's. But now we're getting waaaay off-topic ...

I'm going through more of them now. I moved the history of a couple of the articles to /temp pages partly to see what would happen with watchlists in one case and partly because I'm a control freak in another, but doing that in the "Renee Hartevelt" situation feels super-duper ugly, even to me. Pppery, could you delete that one and then restore just the 2007 edit? Thanks.

Done.

I've been through all the revisions and reimported them or moved them away/ tagged things for speedy deletion where necessary. The only pages that need fixing that I couldn't deal with using these methods were "Talk:Eritrea", "Clearwater River (Idaho)", "Peter Doherty (immunologist)", and "Gefjon". Of those, everything else could theoretically be retrieved from either database dumps or gzinflation/re-encoding except revision 304883915 of "Clearwater River (Idaho)", which would need a Wikimedia sysadmin. I wonder if it'd be possible to write a script to deal with the other cases (perhaps using XML from Special:Export ... in that case, the bytes field for each revision would need to be re-updated), but that's above my programming competence level. I'd be happy to do the import if we go down that route.

Sorry for the stupid question, but why can't those four pages be dealt with the same way you've been dealing with the others?

In T387188#10589728, @Pppery wrote:

Sorry for the stupid question, but why can't those four pages be dealt with the same way you've been dealing with the others?

Not a stupid question (should have clarified earlier) ... the database dumps I have only go up to May 2003.

There's a December 2004 dump at the Internet Archive (which I only know about because it was been brought up in T147146#4760835). That dump specifically wouldn't contain the missing edit to Clearwater River (Idaho) because it's only the old table and that revision would still have been in the cur table when that dump was made.

Indeed re last bit of previous message ... I just figured that out myself. I don't really want to deal with that dump unless I have to ... it's much bigger than the other ones I've used before. Gotta go

But after a closer look it looks like the edits in question were only deleted very temporarily for a history merge, so any dump between when they were made and 2009 (of which many exist on the internet archive would have the content), so it's definitely out there somewhere. But those are even harder to wrangle.

This was a fun expedition to go on

(I'm not expecting any reply to this comment, or planning to pursue this further right now, just jotting down some notes for our future selves)

Nope, the edits in question were deleted in early June 2005 and weren't restored until I got there (hence their high revision ID numbers):
https://en.wikipedia.org/w/index.php?title=Special:Log&logid=211317
https://en.wikipedia.org/w/index.php?title=Special:Log&logid=23807896

Apparently I failed to read the logs correctly then, or conflated two pages - whoops.

perhaps using XML from Special:Export

It turns out that using Special:Export mangles the mojibake further and just gives a replacement character so that's a no-go. I just have to synthesize it using action=raw

I've gone ahead and generated decoded XML files anyway. I just deleted the bytes and sha1 fields from the XML - it imports just fine without them.

Peter Doherty (immunologist)

It turns out that the corrupted revisions here are revdelled as a copyvio, so there's nothing to do here.

Edit: Actually there one revision which was garbled by this bug and not also revdelled as a copyvio. Here's an XML file containing only that one revision. In my opinion it's not worth importing, but for the record:

Gefjon

Talk:Eritrea

I personally would not have undeleted these edits in the first place, even if they weren't corrupt. Make of that what you will.

Clearwater River (Idaho)

Note that this XML is incomplete - it silently doesn't include the revisions that had db:// urls. One of those is the first revision, and the other is from December 2004 in the middle. So beware ..

(If any sysadmin sees this and wants to investigate, we would be interested in seeing what, if any, text the external store URL DB://cluster1/9458279 points to to conclude this)

The code I used to generate these is at P74013

In T387188#10598497, @Pppery wrote:

perhaps using XML from Special:Export

It turns out that using Special:Export mangles the mojibake further and just gives a replacement character so that's a no-go. I just have to synthesize it using action=raw

Of course it'd do that ... silly me!

Thanks very much for making the XML files; I've imported them and wangled the original edits out, etc., along with the 2002 edit for Clearwater River (idaho) from the old database dump. I don't think this runtime error should be seen at all by regular Wikipedia users.

It's occurred to me that the missing 2004 edit at Clearwater River (idaho) might be irretrievable, as the number given seems like it refers to an old revision ID. But who knows?

I've gone ahead and revdelled the db:// edit at Clearwater River (Idaho) since I think it's less confusing to see a revdelled edit (a well known signal that there was some content here but you can't see it) than the alternative. Which makes everything that can be done by either of us.

Oh yeah, I almost forgot to ask: why did the XML files you generated have text like <ns0:revision> instead of <revision> like I've encountered elsewhere?

Because I was using Python's XML library to process the XML, and it does that. See https://github.com/python/cpython/issues/113425

(I downloaded the 2004 data dump to work on something else, and just to be 100% sure, I checked this case too - the missing edit to "Clearwater River (Idaho)" is not in that dump)

This is no longer a production error. It's still a valid DB corruption issue that a sysadmin should investigate if possible (probably by running https://www.mediawiki.org/wiki/Manual:FetchText.php)