Page MenuHomePhabricator

RebuildItemTerms (and probably RebuildPropertyTerms) explodes when there's a (double) redirect
Closed, ResolvedPublic

Description

Problem
Double redirects, if they exist at the moment of rebuilding the new terms store will cause the script to fatal, as those double redirects are not being handled properly (both proeprty and item terms rebuilders).

Solution
Double redirects need to be skipped. Only the final destination item's term need to be moved to the store.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 5 2019, 12:44 PM

There's literally a TODO for this :)

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptAug 5 2019, 2:32 PM

Change 528160 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Ignore in case of UnresolvedEntityRedirect in ItemTermsRebuilder

https://gerrit.wikimedia.org/r/528160

Change 528160 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Ignore in case of UnresolvedEntityRedirect in ItemTermsRebuilder

https://gerrit.wikimedia.org/r/528160

Change 528514 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Fix the ItemTermsRebuilder exception for redirects

https://gerrit.wikimedia.org/r/528514

Change 528514 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fix the ItemTermsRebuilder exception for redirects

https://gerrit.wikimedia.org/r/528514

This doesn't fix the issue. When testing the updated script with double-redirects, it still failed due to the actual exceptipon being thrown is ItemLookupException and not RevisionedUnresolvedRedirectException.

Wikibase\DataModel\Services\Lookup\ItemLookupException from line 33 of /var/www/mediawiki/vendor/wikibase/data-model-services/src/Lookup/LegacyAdapterItemLookup.php: Unresolved redirect from Q2428 to Q2429
#0 /var/www/mediawiki/extensions/Wikibase/repo/includes/Store/ItemTermsRebuilder.php(109): Wikibase\DataModel\Services\Lookup\LegacyAdapterItemLookup->getItemForId(Object(Wikibase\DataModel\Entity\ItemId))
#1 /var/www/mediawiki/extensions/Wikibase/repo/includes/Store/ItemTermsRebuilder.php(73): Wikibase\Repo\Store\ItemTermsRebuilder->rebuildTermsForBatch(Array)
#2 /var/www/mediawiki/extensions/Wikibase/repo/maintenance/rebuildItemTerms.php(84): Wikibase\Repo\Store\ItemTermsRebuilder->rebuild()
#3 /var/www/mediawiki/maintenance/doMaintenance.php(99): Wikibase\RebuildItemTerms->execute()
#4 /var/www/mediawiki/extensions/Wikibase/repo/maintenance/rebuildItemTerms.php(123): require_once('/var/www/mediaw...')
#5 {main}
Wikibase\Lib\Store\RevisionedUnresolvedRedirectException from line 152 of /var/www/mediawiki/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntityRevisionLookup.php: Unresolved redirect from Q2428 to Q2429
#0 /var/www/mediawiki/extensions/Wikibase/lib/includes/Store/DispatchingEntityRevisionLookup.php(59): Wikibase\Lib\Store\Sql\WikiPageEntityRevisionLookup->getEntityRevision(Object(Wikibase\DataModel\Entity\ItemId), 0, 'replica')
#1 /var/www/mediawiki/extensions/Wikibase/lib/includes/Store/TypeDispatchingEntityRevisionLookup.php(54): Wikibase\Lib\Store\DispatchingEntityRevisionLookup->getEntityRevision(Object(Wikibase\DataModel\Entity\ItemId), 0, 'replica')
#2 /var/www/mediawiki/extensions/Wikibase/lib/includes/Store/CacheRetrievingEntityRevisionLookup.php(75): Wikibase\Lib\Store\TypeDispatchingEntityRevisionLookup->getEntityRevision(Object(Wikibase\DataModel\Entity\ItemId), 0, 'replica')
#3 /var/www/mediawiki/extensions/Wikibase/lib/includes/Store/RevisionBasedEntityLookup.php(39): Wikibase\Lib\Store\CacheRetrievingEntityRevisionLookup->getEntityRevision(Object(Wikibase\DataModel\Entity\ItemId))
#4 /var/www/mediawiki/extensions/WikibaseLexeme/vendor/wikibase/data-model-services/src/Lookup/RedirectResolvingEntityLookup.php(53): Wikibase\Lib\Store\RevisionBasedEntityLookup->getEntity(Object(Wikibase\DataModel\Entity\ItemId))
#5 /var/www/mediawiki/vendor/wikibase/data-model-services/src/Lookup/LegacyAdapterItemLookup.php(31): Wikibase\DataModel\Services\Lookup\RedirectResolvingEntityLookup->getEntity(Object(Wikibase\DataModel\Entity\ItemId))
#6 /var/www/mediawiki/extensions/Wikibase/repo/includes/Store/ItemTermsRebuilder.php(109): Wikibase\DataModel\Services\Lookup\LegacyAdapterItemLookup->getItemForId(Object(Wikibase\DataModel\Entity\ItemId))
#7 /var/www/mediawiki/extensions/Wikibase/repo/includes/Store/ItemTermsRebuilder.php(73): Wikibase\Repo\Store\ItemTermsRebuilder->rebuildTermsForBatch(Array)
#8 /var/www/mediawiki/extensions/Wikibase/repo/maintenance/rebuildItemTerms.php(84): Wikibase\Repo\Store\ItemTermsRebuilder->rebuild()
#9 /var/www/mediawiki/maintenance/doMaintenance.php(99): Wikibase\RebuildItemTerms->execute()
#10 /var/www/mediawiki/extensions/Wikibase/repo/maintenance/rebuildItemTerms.php(123): require_once('/var/www/mediaw...')
#11 {main}

Suggestion here:

  • catch ItemLookupException and report to output those items that fail to rebuild (with exception msg); or
  • fail on all exception so that we fix or decide what to do case-by-case
alaa_wmde triaged this task as High priority.Aug 12 2019, 12:31 PM

This will block migration of item terms to new store

This doesn't fix the issue. When testing the updated script with double-redirects, it still failed due to the actual exceptipon being thrown is ItemLookupException and not RevisionedUnresolvedRedirectException.

That's not correct, I got that in beta cluster while doing the migration.

I think we should just ignore anything that can't be read or written.

Change 532777 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Ignore ItemLookupException in ItemTermsRebuilder

https://gerrit.wikimedia.org/r/532777

Change 532777 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Ignore ItemLookupException in ItemTermsRebuilder

https://gerrit.wikimedia.org/r/532777

WMDE-leszek closed this task as Resolved.Sep 11 2019, 11:25 AM