This happens for Q4941387, which currently has a statement with (non-existing) P107 for which "The value is invalid and cannot be displayed.". This seems like the probable cause for this.

Example: https://www.wikidata.org/wiki/Special:EntityData/Q4941387.ttl?flavor=dump&revision=2305050347

Prio Notes:

In the task time meeting, @ArthurTaylor, @hoo and myself would prioritize this above the other tech tasks currently in the Unified DOT Backlog, but the overall prioritization remains to be decided together with product.

In T384625#10544194, @hoo wrote:

This happens for Q4941387, which currently has a statement with (non-existing) P107 for which "The value is invalid and cannot be displayed.". This seems like the probable cause for this.

Example: https://www.wikidata.org/wiki/Special:EntityData/Q4941387.ttl?flavor=dump&revision=2305050347

Which would make this kinda similar to T372993: Resolve type errors caused by data type changes in test Wikidata , I think.

The request ID in the original task description corresponds to Q2772307, which also had a P107 statement at the time (but was already removed since then).

Potentially related, the last ttl dump failed with:

LogicException from line 523 of /srv/mediawiki/php-1.44.0-wmf.15/vendor/wikimedia/purtle/src/RdfWriterBase.php: Bad transition: 10 -> 5
#0 /srv/mediawiki/php-1.44.0-wmf.15/vendor/wikimedia/purtle/src/RdfWriterBase.php(278): Wikimedia\Purtle\RdfWriterBase->state(5)
#1 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Rdf/RdfBuilder.php(131): Wikimedia\Purtle\RdfWriterBase->finish()
#2 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Dumpers/RdfDumpGenerator.php(86): Wikibase\Repo\Rdf\RdfBuilder->finishDocument()
#3 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php(271): Wikibase\Repo\Dumpers\RdfDumpGenerator->postDump()
#4 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/maintenance/DumpEntities.php(233): Wikibase\Repo\Dumpers\DumpGenerator->generateDump(Object(Wikibase\Repo\Store\Sql\SqlEntityIdPager))
#5 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/maintenance/dumpRdf.php(102): Wikibase\Repo\Maintenance\DumpEntities->execute()
#6 /srv/mediawiki/php-1.44.0-wmf.15/maintenance/includes/MaintenanceRunner.php(695): Wikibase\Repo\Maintenance\DumpRdf->execute()
#7 /srv/mediawiki/php-1.44.0-wmf.15/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#8 /srv/mediawiki/multiversion/MWScript.php(156): require_once('/srv/mediawiki/...')
#9 {main}

Special:WhatLinksHere/Property:P107 shows no results in the main namespace, which is suspicious (possibly a separate issue), but haswbstatement:P107 works and shows that two items (Q1534153 and the aforementioned Q4941387) link to this particular deleted property.

@Ifrahkhanyaree_WMDE Can you prioritize this highly? Dumps are not being generated because of this already.

Prpl doesn't have any input on this yet because we haven't had an onboarding to this yet. From the cursory glance the team gave this, it doesn't seem to be directly a Dumps thing but further up causing it to trickle down? And thus might be something more on WD team's plate.

Makes sense.
@Arian_Bozorg, @karapayneWMDE: Can you prioritize this? Reusers are not getting dumps and this is pretty bad.

It took a moment, but I figured out the steps to reproduce this locally:

1. Create a property of data type Item. (Some other data types probably also work, but not all; in particular, String does not work.)
2. Create an item.
3. On the item, create a statement with that property linking to an item (e.g. itself).
4. Add a sitelink to the item. (This is important!) For instance, you can create the item’s talk page, then use that as its sitelink for the local wiki.
5. Delete the property.
6. Try to load the item’s TTL Special:EntityData.

With the sitelink, the sequence of RDF writes is then roughly:

1. Set item as subject, emit “a Item” type. (State is now “object”.)
2. Change subject to sitelink, emit “schema:about item” triple and other sitelink info. (State is now “object”.)
3. Preparing for truthy statement, change subject back to item. (State is now “subject”.)
4. Try to emit truthy triple for statement. (This would change the state to “object.”) This crashes fails. (State remains “subject”.)
5. Try to emit full triples for statement. This starts by changing subject back to item, which crashes – this is the bad transition.

Whereas without the sitelink, you instead get:

1. Set item as subject, emit “a Item” type. (State is now “object”.)
2. Preparing for truthy statement, change subject back to item. However, Purtle is smart enough to notice that the subject hasn’t changed, and so stays in “object” state – this is how it can emit nice semicolon-separated triples rather than full triples all the time.
3. Try to emit truthy triple for statement. This crashes. State remains “object”.
4. Try to emit full triples for statement. This works just fine.

Change #1121354 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Fix bad state transition on unknown snak type

https://gerrit.wikimedia.org/r/1121354

An additional note on the last comment: if I’m not mistaken, the crash can only happen if the statement with the deleted property is the first statement on the item, which might further explain why this didn’t blow up the dumps all the time.

Change #1121354 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Fix bad state transition on unknown snak type

https://gerrit.wikimedia.org/r/1121354

Change #1121376 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] Improve bad state transition behavior

https://gerrit.wikimedia.org/r/1121376

Change #1122143 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@wmf/1.44.0-wmf.17] Fix bad state transition on unknown snak type

https://gerrit.wikimedia.org/r/1122143

Change #1122143 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@wmf/1.44.0-wmf.17] Fix bad state transition on unknown snak type

https://gerrit.wikimedia.org/r/1122143

Mentioned in SAL (#wikimedia-operations) [2025-02-24T14:26:48Z] <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1122143|Fix bad state transition on unknown snak type (T384625)]]

Mentioned in SAL (#wikimedia-operations) [2025-02-24T14:30:31Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde: Backport for [[gerrit:1122143|Fix bad state transition on unknown snak type (T384625)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-02-24T14:37:20Z] <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1122143|Fix bad state transition on unknown snak type (T384625)]] (duration: 10m 31s)

Change #1122149 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] WIP: Test bad state transition behavior

https://gerrit.wikimedia.org/r/1122149

Change #1121376 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Improve bad state transition behavior

https://gerrit.wikimedia.org/r/1121376

Change #1123378 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] POC: Catch unknown errors when dumping entities

https://gerrit.wikimedia.org/r/1123378

In T384625#10544269, @Lucas_Werkmeister_WMDE wrote:

This (and a probably-related issue involving a transition from 10 -> 5, i.e. STATE_DOCUMENT), is breaking this week’s dumps. The dumps are pretty important, so let’s call this High priority (possibly even UBN?).

(In hindsight, it clearly should’ve been UBN! after all.)

Change #1123378 abandoned by Lucas Werkmeister (WMDE):

[mediawiki/extensions/Wikibase@master] POC: Catch unknown errors when dumping entities

Reason:

turns out there’s a whole ExceptionHandler infrastructure for this already

https://gerrit.wikimedia.org/r/1123378

In T384625#10544272, @hoo wrote:

Potentially related, the last ttl dump failed with:

LogicException from line 523 of /srv/mediawiki/php-1.44.0-wmf.15/vendor/wikimedia/purtle/src/RdfWriterBase.php: Bad transition: 10 -> 5
#0 /srv/mediawiki/php-1.44.0-wmf.15/vendor/wikimedia/purtle/src/RdfWriterBase.php(278): Wikimedia\Purtle\RdfWriterBase->state(5)
#1 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Rdf/RdfBuilder.php(131): Wikimedia\Purtle\RdfWriterBase->finish()
#2 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Dumpers/RdfDumpGenerator.php(86): Wikibase\Repo\Rdf\RdfBuilder->finishDocument()
#3 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/includes/Dumpers/DumpGenerator.php(271): Wikibase\Repo\Dumpers\RdfDumpGenerator->postDump()
#4 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/maintenance/DumpEntities.php(233): Wikibase\Repo\Dumpers\DumpGenerator->generateDump(Object(Wikibase\Repo\Store\Sql\SqlEntityIdPager))
#5 /srv/mediawiki/php-1.44.0-wmf.15/extensions/Wikibase/repo/maintenance/dumpRdf.php(102): Wikibase\Repo\Maintenance\DumpEntities->execute()
#6 /srv/mediawiki/php-1.44.0-wmf.15/maintenance/includes/MaintenanceRunner.php(695): Wikibase\Repo\Maintenance\DumpRdf->execute()
#7 /srv/mediawiki/php-1.44.0-wmf.15/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
#8 /srv/mediawiki/multiversion/MWScript.php(156): require_once('/srv/mediawiki/...')
#9 {main}

@hoo do you know if this actually killed the dumps or not? Because to me it looks like the code should have handled this gracefully:

• DumpGenerator::dumpEntities() wraps each entity dumping into a try+catch block for EntityLookupException | StorageException | LogicException (i.e., including the exception we see here)
• If the exception is caught, it calls $this->exceptionHandler->handleException() and then implicitly continues with the next entity if the exception handler doesn’t itself throw
• DumpGenerator::__construct() sets the default exception handler to a new RethrowingExceptionHandler which does what it says on the tin, but:
• DumpEntities::execute() (the base method of the dumper maintenance scripts) sets the dump generator’s handler to a new ReportingExceptionHandler
• ReportingExceptionHandler writes exceptions to a given MessageReporter (which ends up in the maintenance script’s logMessage() method) and then does nothing

So it should have logged the exception (sadly only to the maintenance script’s stderr rather than logstash, probably) but not killed the whole dump run… but then why were the dumps still missing, as reported at T386401?

Oh, probably because RdfDumpGenerator reuses the same RdfBuilder throughout its lifetime, so it would still be in the bad state even if it continues with the next item. (That also explains why the bad transition is 10 -> 5 rather than 10 -> 10 – 5 is the “document” state, so we’re seeing it trying to start the document for the next entity, and failing.)

Change #1123401 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[mediawiki/extensions/Wikibase@master] POC: Handle broken RDF dump generation better

https://gerrit.wikimedia.org/r/1123401

In T384625#10587353, @Lucas_Werkmeister_WMDE wrote:

… to me it looks like the code should have handled this gracefully:

• …

So it should have logged the exception (sadly only to the maintenance script’s stderr rather than logstash, probably) but not killed the whole dump run…

In T384625#10587367, @Lucas_Werkmeister_WMDE wrote:

Oh, probably because RdfDumpGenerator reuses the same RdfBuilder throughout its lifetime, so it would still be in the bad state even if it continues with the next item. (That also explains why the bad transition is 10 -> 5 rather than 10 -> 10 – 5 is the “document” state, so we’re seeing it trying to start the document for the next entity, and failing.)

Now tracked at T389058: Make RDF dumps more resiliant against state transition errors.

Change #1122149 abandoned by Lucas Werkmeister (WMDE):

[mediawiki/extensions/Wikibase@master] WIP: Test bad state transition behavior

https://gerrit.wikimedia.org/r/1122149

I'm having a little trouble following the thread. It seems this bug is listed as high. Is there a sense of the timeframe/how challenging it will be to correct?