Maniphest T219242

deployment-mediawiki-09 PHP7 has broken cache of codebase?
Closed, DeclinedPublicBUG REPORT
Actions

Assigned To

Authored By

	Krenair
	Mar 25 2019, 11:15 PM

Description

From https://en.wikipedia.beta.wmflabs.org/w/api.php while I am logged in, but not while I am logged out:

[XJlgAqwQBGoAAFPjDhoAAAAD] /w/api.php Wikimedia\Services\NoSuchServiceException from line 451 of /srv/mediawiki/php-master/includes/libs/services/ServiceContainer.php: No such service: MobileFrontend.Context
Backtrace:
#0 /srv/mediawiki/php-master/includes/libs/services/ServiceContainer.php(414): Wikimedia\Services\ServiceContainer->createService(string)
#1 /srv/mediawiki/php-master/extensions/MobileFrontend/includes/MobileFrontend.skin.hooks.php(182): Wikimedia\Services\ServiceContainer->getService(string)
#2 /srv/mediawiki/php-master/extensions/MobileFrontend/includes/MobileFrontend.hooks.php(178): MobileFrontendSkinHooks::prepareFooter(SkinApi, SkinApiTemplate)
#3 /srv/mediawiki/php-master/includes/Hooks.php(174): MobileFrontendHooks::onSkinTemplateOutputPageBeforeExec(SkinApi, SkinApiTemplate)
#4 /srv/mediawiki/php-master/includes/Hooks.php(202): Hooks::callHook(string, array, array, NULL)
#5 /srv/mediawiki/php-master/includes/skins/SkinTemplate.php(481): Hooks::run(string, array)
#6 /srv/mediawiki/php-master/includes/skins/SkinTemplate.php(225): SkinTemplate->prepareQuickTemplate()
#7 /srv/mediawiki/php-master/includes/OutputPage.php(2709): SkinTemplate->outputPage()
#8 /srv/mediawiki/php-master/includes/api/ApiFormatBase.php(332): OutputPage->output()
#9 /srv/mediawiki/php-master/includes/api/ApiFormatRaw.php(85): ApiFormatBase->closePrinter()
#10 /srv/mediawiki/php-master/includes/api/ApiMain.php(1856): ApiFormatRaw->closePrinter()
#11 /srv/mediawiki/php-master/includes/api/ApiMain.php(598): ApiMain->printResult(integer)
#12 /srv/mediawiki/php-master/includes/api/ApiMain.php(538): ApiMain->handleException(Wikimedia\Services\NoSuchServiceException)
#13 /srv/mediawiki/php-master/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#14 /srv/mediawiki/php-master/api.php(87): ApiMain->execute()
#15 /srv/mediawiki/w/api.php(3): require(string)
#16 {main}

Details

	Subject	Repo	Branch	Lines +/-
	scap: Make wmflabs php7 behaviour match prod's	operations/puppet	production	+2 -0

Customize query in gerrit

Related Objects

Mentioned In: T135427: Beta puppetmaster cherry-pick process
Mentioned Here: T221347: PHP7 opcache sometimes corrupts when cleared (was: Fatal ConfigException, undefined InitialiseSettings variable)
T224491: PHP 7 corruption during deployment (was: PHP 7 fatals on mw1262)
rEMFR95629c2ef014: Move creation of MobileContext to MediaWikiServices

Event Timeline

Krenair triaged this task as Unbreak Now! priority.Mar 25 2019, 11:15 PM

Krenair created this task.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptMar 25 2019, 11:15 PM

Krenair renamed this task from API broken on beta due to missing MobileFrontend.Context service to API broken on beta for unknown number of logged in users (?) due to missing MobileFrontend.Context service.Mar 25 2019, 11:17 PM

Krenair updated the task description. (Show Details)

Jdforrester-WMF removed projects: Release, Train Deployments.Mar 25 2019, 11:18 PM

Jdforrester-WMF changed the subtype of this task from "Task" to "Bug Report".

@Jdforrester-WMF also sees this while logged in but @Paladox does not. Not seen while logged out

This was added about 2 weeks ago, so not sure why it's only surfacing now > https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/494825/
@pmiazga when you are done with the other UBN could you investigate?

The MobileFrontend.Context is available for 12 days in the codebase (https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/blob/master/includes/ServiceWiring.php#L52), it was added in rEMFR95629c2ef014: Move creation of MobileContext to MediaWikiServices.

Is it possible that this patch (https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MobileFrontend/+/494825/) somehow got lost/disconnected somewhere?

/cc @Krenair

Specifically this is in beta not prod so I assumed it to be a new issue, checking

I checked the https://en.wikipedia.beta.wmflabs.org/w/api.php and it works for me bot as logged in and logged out user.

It seems this is coming up when a user hits the deployment-mediawiki-09 server with the PHP_ENGINE=php7 cookie

Krenair edited projects, added Beta-Cluster-Infrastructure; removed Web-Team-Backlog, MobileFrontend.Mar 25 2019, 11:35 PM

Maybe it's a opcache issue?

Krenair renamed this task from API broken on beta for unknown number of logged in users (?) due to missing MobileFrontend.Context service to deployment-mediawiki-09 PHP7 has broken cache of codebase?.Mar 25 2019, 11:36 PM

Krenair removed a parent task: T206677: 1.33.0-wmf.23 deployment blockers.

I remembered a thing @Joe mentioned about admin ports when there was an opcache issue like this recently, have a patch that seems to have done the trick after a puppet run + scap pull on the misbehaving box

Change 499025 had a related patch set uploaded (by Alex Monk; owner: Alex Monk):
[operations/puppet@production] scap: Make wmflabs php7 behaviour match prod's

https://gerrit.wikimedia.org/r/499025

gerritbot added a project: Patch-For-Review.Mar 25 2019, 11:52 PM

greg unsubscribed.Mar 27 2019, 11:23 PM

Krinkle updated the task description. (Show Details)May 29 2019, 3:58 PM

hashar unsubscribed.May 30 2019, 8:28 PM

Krinkle closed this task as Declined.Mar 17 2020, 1:32 AM

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptMar 17 2020, 1:32 AM

taavi mentioned this in T135427: Beta puppetmaster cherry-pick process.Apr 12 2021, 2:47 PM