Page MenuHomePhabricator

[WBQC] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase
Open, Needs TriagePublic

Description

On this Gerrit change, @Michael reported that running

php maintenance/run.php WikibaseQualityConstraints:ImportConstraintEntities.php

fails on a Wikibase with default settings, with an EntityContentTooBigException: On Wikidata, the maxSerializedEntitySize is 3000 (kibibytes), but by default it’s $wgMaxArticleSize, which in turn is only 2048 kiB. The serialized entity size is also checked during deserialization, so while ImportConstraintEntities removes statements, they can still cause the error:

importEntityFromJson()
		$wikidataEntityArray = json_decode( $wikidataEntitiesJson, true )['entities'][$wikidataEntityId];
		$wikidataEntity = $this->entityDeserializer->deserialize( $wikidataEntityArray ); // <-- EntityContentTooBigException thrown here

		$wikidataEntity->setId( null );

		if ( $wikidataEntity instanceof StatementListProvider ) {
			$wikidataEntity->getStatements()->clear();
		}
		// entity would *now* be small enough to fit in the local wiki (probably)

TODO: Check which entity is causing the error, and if it’s still too large without the statements. If not, then a relatively simple solution would be to unset( $wikidataEntityArray['claims'] ) before deserializing the entity.

Event Timeline

Adding to the wikidata-tech board – I propose that this has impact on development efforts and/or onboarding efforts. (Developers who want to work with the extension should run this maintenance script, to populate their local wiki with the needed entities.)

Hm, I tried reproducing this by commenting out all my $wgWBQualityConstraints...Id = '...'; settings in LocalSettings (so that the script would try to import them again), and then running the script in dry-run mode:

php maintenance/run.php WikibaseQualityConstraints:ImportConstraintEntities --dry-run

But I didn’t get any errors then. Perhaps the error only happens when actually saving the entities after all?

ItamarWMDE renamed this task from ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase to [Q-CON] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase.Sep 20 2023, 2:29 PM
ItamarWMDE renamed this task from [Q-CON] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase to [WDQC] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase.Sep 20 2023, 3:01 PM
ItamarWMDE renamed this task from [WDQC] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase to [WBQC] ImportConstraintEntities:ImportConstraintEntities maintenance script fails in a default Wikibase.Nov 8 2023, 10:52 AM