Maniphest T204797

[Regression 1.32.0-wmf.22] ParserOutput::getLanguageLinks returns invalid values (Undefined index from ApiParse and LinksUpdate)
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Krinkle
	Sep 19 2018, 2:51 AM

Description

ApiParse Error

Request ID: W6G4wwrAIGEAALMp8yIAAADB

message

[{exception_id}] {exception_url}   ErrorException from line 668 of /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiParse.php: PHP Notice: Undefined index: 1

stacktrace

#0 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiParse.php(668): MWExceptionHandler::handleError(integer, string, string, integer, array, array)
#1 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiParse.php(371): ApiParse->formatLangLinks(array)
#2 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiMain.php(1587): ApiParse->execute()
#3 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiMain.php(531): ApiMain->executeAction()
#4 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#5 /srv/mediawiki/php-1.32.0-wmf.22/api.php(87): ApiMain->execute()

Source: https://gerrit.wikimedia.org/g/mediawiki/core/+/6df38a8d26681d5241e2f7808ee3ed69472bb329/includes/api/ApiParse.php#649

	private function formatLangLinks( $links ) { /* */
		foreach ( $links as $link ) { /* */
			$bits = explode( ':', $link, 2 );  /* */
			ApiResult::setContentValue( $entry, 'title', $bits[1] );

LinksUpdate error

trace

PHP Notice: Undefined index: 1

#0 /srv/mediawiki/php-1.32.0-wmf.22/includes/deferred/LinksUpdate.php(140): MWExceptionHandler::handleError(integer, string, string, integer, array, array)
#1 /srv/mediawiki/php-1.32.0-wmf.22/includes/content/AbstractContent.php(238): LinksUpdate->__construct(Title, ParserOutput, boolean)
#2 /srv/mediawiki/php-1.32.0-wmf.22/includes/Storage/DerivedPageDataUpdater.php(1313): AbstractContent->getSecondaryDataUpdates(Title, NULL, boolean, ParserOutput)
#3 /srv/mediawiki/php-1.32.0-wmf.22/includes/Storage/DerivedPageDataUpdater.php(1549): MediaWiki\Storage\DerivedPageDataUpdater->getSecondaryDataUpdates(boolean)
#4 /srv/mediawiki/php-1.32.0-wmf.22/includes/Storage/DerivedPageDataUpdater.php(1393): MediaWiki\Storage\DerivedPageDataUpdater->doSecondaryDataUpdates(array)
#5 /srv/mediawiki/php-1.32.0-wmf.22/includes/Storage/PageUpdater.php(1193): MediaWiki\Storage\DerivedPageDataUpdater->doUpdates()
#6 /srv/mediawiki/php-1.32.0-wmf.22/includes/libs/rdbms/database/Database.php(3746): Closure$MediaWiki\Storage\PageUpdater::getAtomicSectionUpdate(Wikimedia\Rdbms\DatabaseMysqli, string)
#7 /srv/mediawiki/php-1.32.0-wmf.22/includes/libs/rdbms/database/DBConnRef.php(49): Wikimedia\Rdbms\Database->doAtomicSection(string, Closure$MediaWiki\Storage\PageUpdater::getAtomicSectionUpdate;2898)
#8 /srv/mediawiki/php-1.32.0-wmf.22/includes/libs/rdbms/database/DBConnRef.php(529): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#9 /srv/mediawiki/php-1.32.0-wmf.22/includes/deferred/AtomicSectionUpdate.php(35): Wikimedia\Rdbms\DBConnRef->doAtomicSection(string, Closure$MediaWiki\Storage\PageUpdater::getAtomicSectionUpdate;2898)
#10 /srv/mediawiki/php-1.32.0-wmf.22/includes/deferred/DeferredUpdates.php(268): AtomicSectionUpdate->doUpdate()
#11 /srv/mediawiki/php-1.32.0-wmf.22/includes/deferred/DeferredUpdates.php(226): DeferredUpdates::runUpdate(AtomicSectionUpdate, Wikimedia\Rdbms\LBFactoryMulti, string, integer)
#12 /srv/mediawiki/php-1.32.0-wmf.22/includes/deferred/DeferredUpdates.php(130): DeferredUpdates::execute(array, string, integer)
#13 /srv/mediawiki/php-1.32.0-wmf.22/includes/MediaWiki.php(607): DeferredUpdates::doUpdates(string, integer)
#14 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiMain.php(548): MediaWiki::preOutputCommit(DerivativeContext)
#15 /srv/mediawiki/php-1.32.0-wmf.22/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#16 /srv/mediawiki/php-1.32.0-wmf.22/api.php(87): ApiMain->execute()
#17 /srv/mediawiki/w/api.php(3): include(string)
#18 {main}

While this code is separate, it is a copy of what ApiParse contains. Depending on the fix will need to either be patched in two places, or both be fixed by the same common source.

Notes

Searching for message:"ApiParse.php" AND message:"Undefined index" shows that this error is newly introduced in 1.32.0-wmf.22. It did not exist previously.

Example url that triggers the above error in the backend: https://www.mediawiki.org/w/api.php?action=parse&page=Help%3ASubpages%2Fhi&format=json

Details

	Subject	Repo	Branch	Lines +/-
	Avoid warnings and errors caused by x-pagetranslation-tag	mediawiki/extensions/Translate	wmf/1.32.0-wmf.22	+3 -1
	Avoid warnings and errors caused by x-pagetranslation-tag	mediawiki/extensions/Translate	master	+3 -1

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved	Release	hashar	T191068 1.32.0-wmf.22 deployment blockers
		Resolved	PRODUCTION ERROR	Nikerabbit	T204797 [Regression 1.32.0-wmf.22] ParserOutput::getLanguageLinks returns invalid values (Undefined index from ApiParse and LinksUpdate)

Event Timeline

Krinkle created this task.Sep 19 2018, 2:51 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 19 2018, 2:51 AM

Krinkle added a parent task: T191068: 1.32.0-wmf.22 deployment blockers.Sep 19 2018, 2:52 AM

This seems like the same root cause as T204756: DerivedPageDataUpdater emits "PHP Notice: Undefined index: 1" from LinksUpdate.php.

Krinkle renamed this task from [Regression 1.32.0-wmf.22] ApiParse causes "PHP Error: Undefined index: 1" to [Regression 1.32.0-wmf.22] ParserOutput::getLanguageLinks returns invalid values (Undefined index from ApiParse and LinksUpdate).Sep 19 2018, 3:17 AM

Krinkle updated the task description. (Show Details)

Krinkle merged a task: T204756: DerivedPageDataUpdater emits "PHP Notice: Undefined index: 1" from LinksUpdate.php.

Krinkle edited projects, added MediaWiki-General, Core-Platform-Team-Old; removed MediaWiki-Action-API.

Krinkle added subscribers: hashar, daniel, greg, Liuxinyu970226.

It turns out the bad value is "x-pagetranslation-tag", added by the Translate extension since rETRAd3935a65cfca: Allow displaying page translation language list in the sidebar.

Apparently it's trying to insert some dummy value into the ParserOutput and then specially handle it in 'LanguageLinks' hook. Unfortunately that breaks code paths that don't call that hook, including ApiParse's output of the links generated by a parse and LinksUpdate's updating of the database langlinks table.

It looks like it's only using it as a flag for one of its modes, so perhaps that purpose could be as well served by using $parserOutput->setExtensionData() instead.

LanguageLinks hook doesn't provide any mechanism for accessing contextual information. I don't see any other way around that other than changing the hook itself.

In Translate's case, the language links should not be stored in the database, because they can change independently of the page. I am not sure though how Wikidata solved this.

Does anyone have ideas how to go about solving this?

Nikerabbit added a project: User-Nikerabbit.Sep 19 2018, 2:34 PM

If there's no quick or obvious way to remediate this, perhaps we should revert the change in question. Then, once the rest of last week's changes have made it to production, we can revisit and take time to find a solution to the problem, perhaps even in time for next week :)

If someone else has a quick or obvious way, please do share (I don't mean to discourage that). I took a look myself and none came to mind. I'm just offering the above (revert option) as a normal path forward and not a sign of failure. I think we should get used to solving issues out of band, instead of during the deployment.

The dummy link could be omitted with the default setting value. That should quiet the warnings.

(Train blocker == UBN!)

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptSep 19 2018, 2:36 PM

Change 461412 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Avoid warnings and errors caused by x-pagetranslation-tag

https://gerrit.wikimedia.org/r/461412

gerritbot added a project: Patch-For-Review.Sep 19 2018, 2:41 PM

Rest of the Language team has likely finished working for today, so outside reviews are welcome if the fix is required today.

In T204797#4597988, @Nikerabbit wrote:

LanguageLinks hook doesn't provide any mechanism for accessing contextual information.

D'oh, you're right.

You could avoid the warning here by adding a colon to the dummy entry, but it'll still put the dummy entry into the database.

In Translate's case, the language links should not be stored in the database, because they can change independently of the page. I am not sure though how Wikidata solved this.

It looks like Wikibase's links do go into the database.

In T204797#4598001, @Krinkle wrote:

If there's no quick or obvious way to remediate this, perhaps we should revert the change in question. Then, once the rest of last week's changes have made it to production, we can revisit and take time to find a solution to the problem, perhaps even in time for next week :)

In T204797#4597855, @Anomie wrote:

It turns out the bad value is "x-pagetranslation-tag", added by the Translate extension since rETRAd3935a65cfca: Allow displaying page translation language list in the sidebar.

So reverting that ^ change, yes? If the patch from Niklas won't be reviewed quickly let's just do that (the revert).

Change 461412 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Avoid warnings and errors caused by x-pagetranslation-tag

https://gerrit.wikimedia.org/r/461412

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-09-25 (1.32.0-wmf.23)).Sep 19 2018, 5:01 PM

• Petar.petkovic removed a project: Patch-For-Review.Sep 19 2018, 6:07 PM

Change 461447 had a related patch set uploaded (by Hashar; owner: Nikerabbit):
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.22] Avoid warnings and errors caused by x-pagetranslation-tag

https://gerrit.wikimedia.org/r/461447

gerritbot added a project: Patch-For-Review.Sep 19 2018, 6:41 PM

Change 461447 merged by jenkins-bot:
[mediawiki/extensions/Translate@wmf/1.32.0-wmf.22] Avoid warnings and errors caused by x-pagetranslation-tag

https://gerrit.wikimedia.org/r/461447

Mentioned in SAL (#wikimedia-operations) [2018-09-19T19:16:37Z] <hashar@deploy1001> Synchronized php-1.32.0-wmf.22/extensions/Translate/tag/PageTranslationHooks.php: Avoid warnings and errors caused by x-pagetranslation-tag - T204797 (duration: 00m 58s)

ReleaseTaggerBot edited projects, added MW-1.32-notes (WMF-deploy-2018-09-18 (1.32.0-wmf.22)); removed MW-1.32-notes (WMF-deploy-2018-09-25 (1.32.0-wmf.23)).Sep 19 2018, 8:00 PM

Thanks!

Krinkle removed a project: MW-1.32-notes (WMF-deploy-2018-09-18 (1.32.0-wmf.22)).Sep 20 2018, 3:00 AM

Liuxinyu970226 unsubscribed.Sep 20 2018, 4:37 AM

Nikerabbit mentioned this in T204914: Improve the way Translate extension injects language links to the interlanguage links section.Sep 20 2018, 7:13 AM

In T204797#4598073, @Anomie wrote:

In Translate's case, the language links should not be stored in the database, because they can change independently of the page. I am not sure though how Wikidata solved this.

It looks like Wikibase's links do go into the database.

In fact, Wikidata causes the page to be re-parsed when the language links change. There's a long standing TODO to optimize this to just load the cached ParserOutput, update the language links, and write it back - but as far as I know, this never happened.

hashar mentioned this in T191068: 1.32.0-wmf.22 deployment blockers.Sep 20 2018, 9:15 PM

Krinkle moved this task from Nov2019/1.35.wmf.5+ to Resolved on the Wikimedia-production-error board.Jan 24 2019, 5:33 AM

• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM

Nikerabbit removed a project: User-Nikerabbit.Mar 25 2020, 12:13 PM

TerraCodes unsubscribed.Mar 26 2020, 4:16 AM

Aklapper removed a subscriber: Anomie.Oct 16 2020, 5:38 PM

Nikerabbit mentioned this in T351866: Fix $wgPageTranslationLanguageList = 'sidebar-fallback'.Nov 23 2023, 7:48 AM