Page MenuHomePhabricator

1.35.0-wmf.39 breaks Flow
Closed, ResolvedPublicPRODUCTION ERROR

Description

Impact

Upon deployment of 1.35.0-wmf.39 Flow ends up broken. Each access to a topic cause one occurence of Messages: $key must be a string or an array, and several occurences of extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php:62
PHP Notice: Undefined index: print

The Flow page ends up being broken cause PHP Notice: Undefined index: print triggers an exception error. An example given on IRC was https://www.mediawiki.org/wiki/Topic:Vp2ezhpldxustuf4

Both traces seems to relate to building the sidebar

Error message
/srv/mediawiki/php-1.35.0-wmf.39/includes/language/Message.php:261
$key must be a string or an array
Stack Trace
#0 /srv/mediawiki/php-1.35.0-wmf.39/includes/GlobalFunctions.php(1223): Message->__construct(integer)
#1 /srv/mediawiki/php-1.35.0-wmf.39/includes/context/RequestContext.php(443): wfMessage(integer)
#2 /srv/mediawiki/php-1.35.0-wmf.39/includes/context/ContextSource.php(169): RequestContext->msg(integer)
#3 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/Skin.php(2212): ContextSource->msg(integer)
#4 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/Skin.php(2348): Skin->makeLink(integer, NULL, array)
#5 /srv/mediawiki/php-1.35.0-wmf.39/skins/Vector/includes/VectorTemplate.php(411): Skin->makeListItem(integer, NULL, array)
#6 /srv/mediawiki/php-1.35.0-wmf.39/skins/Vector/includes/VectorTemplate.php(325): VectorTemplate->getMenuData(string, array, integer)
#7 /srv/mediawiki/php-1.35.0-wmf.39/skins/Vector/includes/VectorTemplate.php(171): VectorTemplate->buildSidebar()
#8 /srv/mediawiki/php-1.35.0-wmf.39/skins/Vector/includes/VectorTemplate.php(199): VectorTemplate->getSkinData()
#9 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/SkinTemplate.php(127): VectorTemplate->execute()
#10 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/SkinTemplate.php(141): SkinTemplate->generateHTML()
#11 /srv/mediawiki/php-1.35.0-wmf.39/includes/OutputPage.php(2616): SkinTemplate->outputPage()
#12 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(993): OutputPage->output(boolean)
#13 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(1006): MediaWiki->{closure}()
#14 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(543): MediaWiki->main()
#15 /srv/mediawiki/php-1.35.0-wmf.39/index.php(53): MediaWiki->run()
#16 /srv/mediawiki/php-1.35.0-wmf.39/index.php(46): wfIndexMain()
#17 /srv/mediawiki/w/index.php(3): require(string)
#18 {main}
Error message
/srv/mediawiki/php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php:62
PHP Notice: Undefined index: print
Stack Trace
#0 /srv/mediawiki/php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php(62): MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.35.0-wmf.39/includes/HookContainer/HookContainer.php(320): ElectronPdfServiceHooks::onSidebarBeforeOutput(SkinVector, array)
#2 /srv/mediawiki/php-1.35.0-wmf.39/includes/HookContainer/HookContainer.php(131): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#3 /srv/mediawiki/php-1.35.0-wmf.39/includes/HookContainer/HookRunner.php(3490): MediaWiki\HookContainer\HookContainer->run(string, array, array)
#4 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/Skin.php(1649): MediaWiki\HookContainer\HookRunner->onSidebarBeforeOutput(SkinVector, array)
#5 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/SkinTemplate.php(432): Skin->buildSidebar()
#6 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/SkinTemplate.php(125): SkinTemplate->prepareQuickTemplate()
#7 /srv/mediawiki/php-1.35.0-wmf.39/includes/skins/SkinTemplate.php(141): SkinTemplate->generateHTML()
#8 /srv/mediawiki/php-1.35.0-wmf.39/includes/OutputPage.php(2616): SkinTemplate->outputPage()
#9 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(993): OutputPage->output(boolean)
#10 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(1006): MediaWiki->{closure}()
#11 /srv/mediawiki/php-1.35.0-wmf.39/includes/MediaWiki.php(543): MediaWiki->main()
#12 /srv/mediawiki/php-1.35.0-wmf.39/index.php(53): MediaWiki->run()
#13 /srv/mediawiki/php-1.35.0-wmf.39/index.php(46): wfIndexMain()
#14 /srv/mediawiki/w/index.php(3): require(string)
#15 {main}
  1. Notes

Event Timeline

hashar created this task.Jun 30 2020, 2:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 30 2020, 2:43 PM
hashar triaged this task as Unbreak Now! priority.Jun 30 2020, 2:43 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptJun 30 2020, 2:43 PM

They seem tightly related and definitely cause StructuredDiscussions to explode. One stacktrace originates from ElectronPdfServiceHooks::onSidebarBeforeOutput

Both stacktraces refers to the building the sidebar code which is in Vector , Electron might not be the root cause.

hashar updated the task description. (Show Details)Jun 30 2020, 2:56 PM
Ammarpad claimed this task.EditedJun 30 2020, 3:23 PM
Ammarpad added a subscriber: Ammarpad.

I think we don't need to revert that. I suspect the print key is not present in flow page for some reasons. Need to check properly though.

Change 608649 had a related patch set uploaded (by Aron Manning; owner: Aron Manning):
[mediawiki/extensions/ElectronPdfService@master] Hotfix: "Undefined index: print"

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ElectronPdfService/ /608649

Tgr added a subscriber: Tgr.Jun 30 2020, 3:30 PM
>>> $t = Title::newFromText('Topic:Vp9cnfpdti9bnda2')
>>> $skin = \MediaWiki\MediaWikiServices::getInstance()->getSkinFactory()->makeSkin('vector')
>>> $skin->getContext()->setTitle($t)
>>> $skin->buildSidebar()

=> [
     "navigation" => [
       [
         "text" => "Main page",
         "href" => "/wiki/Main_Page",
         "id" => "n-mainpage-description",
         "active" => false,
       ],
       [
         "text" => "Contents",
         "href" => "/wiki/Wikipedia:Contents",
         "id" => "n-contents",
         "active" => false,
       ],
       [
         "text" => "Current events",
         "href" => "/wiki/Portal:Current_events",
         "id" => "n-currentevents",
         "active" => false,
       ],
       [
         "text" => "Random article",
         "href" => "/wiki/Special:Random",
         "id" => "n-randompage",
         "active" => false,
       ],
       [
         "text" => "About Wikipedia",
         "href" => "/wiki/Wikipedia:About",
         "id" => "n-aboutsite",
         "active" => false,
       ],
       [
         "text" => "Contact page",
         "href" => "/wiki//wiki/Wikipedia:Contact_us",
         "id" => "n-contactpage",
         "active" => false,
       ],
       [
         "text" => "Donate to Wikipedia",
         "href" => "//wikimediafoundation.org/wiki/Special:Landingcheck?landing_page=WMFJA085&language=en&utm_source=donate&utm_medium=sidebar&utm_campaign=20101204SB002",
         "id" => "n-sitesupport",
         "active" => false,
         "rel" => "nofollow",
       ],
       [
         "text" => "Wikipedia store",
         "href" => "//shop.wikimedia.org",
         "title" => "Visit the Wikipedia store",
         "id" => "n-shoplink",
       ],
     ],
     "SEARCH" => [],
     "interaction" => [
       [
         "text" => "Help",
         "href" => "https://www.mediawiki.org/wiki/Special:MyLanguage/Help:Contents",
         "id" => "n-help",
         "active" => false,
       ],
       [
         "text" => "Community portal",
         "href" => "/wiki/Wikipedia:Community_portal",
         "id" => "n-portal",
         "active" => false,
       ],
       [
         "text" => "Recent changes",
         "href" => "/wiki/Special:RecentChanges",
         "id" => "n-recentchanges",
         "active" => false,
       ],
       [
         "text" => "Upload file",
         "href" => "/wiki/Wikipedia:File_Upload_Wizard",
         "id" => "n-upload",
         "active" => false,
       ],
     ],
     "TOOLBOX" => [
       "whatlinkshere" => [
         "href" => "/wiki/Special:WhatLinksHere/Topic:Vp9cnfpdti9bnda2",
         "id" => "t-whatlinkshere",
       ],
       "recentchangeslinked" => [
         "href" => "/wiki/Special:RecentChangesLinked/Topic:Vp9cnfpdti9bnda2",
         "msg" => "recentchangeslinked-toolbox",
         "id" => "t-recentchangeslinked",
         "rel" => "nofollow",
       ],
       "upload" => [
         "href" => "/wiki/Wikipedia:File_Upload_Wizard",
         "id" => "t-upload",
       ],
       "specialpages" => [
         "href" => "/wiki/Special:SpecialPages",
         "id" => "t-specialpages",
       ],
       "info" => [
         "text" => "Page information",
         "href" => "/w/index.php?title=Topic:Vp9cnfpdti9bnda2&action=info",
         "id" => "t-info",
       ],
     ],
     "LANGUAGES" => [],
     "electronpdfservice-sidebar-portlet-heading" => [
       [
         "text" => "Download as PDF",
         "id" => "electron-print_pdf",
         "href" => "/w/index.php?title=Special:ElectronPdf&page=Topic%3AVp9cnfpdti9bnda2&action=show-download-screen",
       ],
       null,
     ],
   ]

This might just be a debugging artifact, as I'm probably skipping some skin initialization steps, but note the extra null in the end. That's consistent with the first error.

I think we don't need to revert that. I suspect the print key is not present in flow page for some reasons. Need to check properly though.

No need. Above patch will solve it at least in the short term.

Tgr added a comment.Jun 30 2020, 3:33 PM

...that null is caused by the print field not existing in ElectronPdfServiceHooks::onSidebarBeforeOutput(). So the first error is just a side-effect of the second one.

master and wmf/1.35.0-wmf.39 branches are at the same commit. How do you do backporting in this case? Just fast-forward wmf.39?

Tgr added a comment.Jun 30 2020, 3:47 PM

master and wmf/1.35.0-wmf.39 branches are at the same commit. How do you do backporting in this case? Just fast-forward wmf.39?

They won't be at the same commit once the fix gets merged.

Demian added a comment.EditedJun 30 2020, 3:48 PM

Do I submit a separate patch for that branch, or it can be fast-forwarded (rebased) to the same commit?

Do I submit a separate patch for that branch, or it can be fast-forwarded (rebased) to the same commit?

Submit your fix to master and then it can be backported to wmf.39

Change 608485 had a related patch set uploaded (by Aron Manning; owner: Aron Manning):
[mediawiki/extensions/ElectronPdfService@wmf/1.35.0-wmf.39] Hotfix: "Undefined index: print"

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ElectronPdfService/ /608485

Tgr added a comment.Jun 30 2020, 3:56 PM

Do I submit a separate patch for that branch, or it can be fast-forwarded (rebased) to the same commit?

You can cherry-pick it to another branch via the Gerrit UI. I don't think there's a similarly easy way for fast-forwarding (you could redefine the branch tip but that requires advanced privileges).

Demian added a comment.EditedJun 30 2020, 3:56 PM

Thanks! Done.
Thanks @RhinosF1 for the quick notice.

Change 608649 merged by jenkins-bot:
[mediawiki/extensions/ElectronPdfService@master] Hotfix: "Undefined index: print"

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ElectronPdfService/ /608649

Tgr added a comment.Jun 30 2020, 4:24 PM

Still broken, but maybe that's just due to the sidebar cache?

Still broken, but maybe that's just due to the sidebar cache?

Another thread is working (https://en.wikipedia.beta.wmflabs.org/wiki/Topic:Vp9iok8ssazd6a62). I'd say it is due to caching.

Tgr added a comment.Jun 30 2020, 5:00 PM

@Demian do you want to deploy the backport? The next window is in an hour. See https://wikitech.wikimedia.org/wiki/Backport_windows for docs.
(I can do it if aren't interested.)

Sure, thanks for the link

I'm not on IRC, is that a hard requirement in this case? If so, I'd like to ask you.

Tgr added a comment.Jun 30 2020, 5:15 PM

Yeah, deployments are coordinated on IRC. Will do then. Thanks for writing the patch!

Thank you for all the investigation! I was in meeting and I am heading to dinner.

I will be back at 19:00 UTC which is the slot "Mediawiki train - European+American Version (secondary timeslot)"

Change 608485 merged by jenkins-bot:
[mediawiki/extensions/ElectronPdfService@wmf/1.35.0-wmf.39] Hotfix: "Undefined index: print"

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ElectronPdfService/ /608485

Mentioned in SAL (#wikimedia-operations) [2020-06-30T18:23:17Z] <tgr@deploy1001> Synchronized php-1.35.0-wmf.39/extensions/ElectronPdfService/src/ElectronPdfServiceHooks.php: Backport: [[gerrit:608485|Hotfix: "Undefined index: print" (T256761)]] (duration: 01m 05s)

Ammarpad closed this task as Resolved.Jun 30 2020, 6:24 PM
Ammarpad reassigned this task from Ammarpad to Demian.

Working on testwiki: https://test.wikipedia.org/wiki/Topic:Vp9iyl4amw7ofe4n

Thanks everyone who worked on solving this!

I am impressed :] I will promote the wikis again in a minute. Thank you very much.