startupregistrystats-testwiki fails to run on php-1.44.0-wmf.21
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Clement_Goubert
	Mar 18 2025, 10:36 AM

Description

Error

mwversion: 1.44.0-wmf.21

trace

Mar 18 10:30:15 mwmaint2002 systemd[1]: Starting MediaWiki periodic job startupregistrystats-testwiki...                                                               
Mar 18 10:30:15 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: Checking 1070 modules...                                                               
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: [a15d50f88be1ee3b6485a968] [no req]   Error: Unsupported operand types                 
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: Backtrace:                                                                             
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: from /srv/mediawiki/php-1.44.0-wmf.21/includes/language/Language.php(438)              
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #0 /srv/mediawiki/php-1.44.0-wmf.21/includes/language/Language.php(515): MediaWiki\Language\Language->getNamespaces()
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #1 /srv/mediawiki/php-1.44.0-wmf.21/includes/title/TitleFormatter.php(229): MediaWiki\Language\Language->getNsText(int)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #2 /srv/mediawiki/php-1.44.0-wmf.21/includes/title/TitleFormatter.php(90): MediaWiki\Title\TitleFormatter->getNamespaceName(int, string)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #3 /srv/mediawiki/php-1.44.0-wmf.21/includes/cache/LinkCache.php(115): MediaWiki\Title\TitleFormatter->formatTitle(int, string)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #4 /srv/mediawiki/php-1.44.0-wmf.21/includes/cache/LinkCache.php(239): MediaWiki\Cache\LinkCache->getCacheKey(array, bool)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #5 /srv/mediawiki/php-1.44.0-wmf.21/includes/page/PageStore.php(179): MediaWiki\Cache\LinkCache->isBadLink(array)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #6 /srv/mediawiki/php-1.44.0-wmf.21/includes/page/PageStore.php(156): MediaWiki\Page\PageStore->getPageByNameViaLinkCache(int, string, int)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #7 /srv/mediawiki/php-1.44.0-wmf.21/includes/page/PageStore.php(125): MediaWiki\Page\PageStore->getPageByName(int, string, int)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #8 /srv/mediawiki/php-1.44.0-wmf.21/includes/page/PageStore.php(259): MediaWiki\Page\PageStore->getPageForLink(MediaWiki\Title\TitleValue, int)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #9 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/WikiModule.php(195): MediaWiki\Page\PageStore->getPageByText(string)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #10 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/WikiModule.php(300): MediaWiki\ResourceLoader\WikiModule->getContent(string, MediaWiki\ResourceLoader\DerivativeContext)                                                                          
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #11 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/Module.php(759): MediaWiki\ResourceLoader\WikiModule->getScript(MediaWiki\ResourceLoader\DerivativeContext)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #12 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/Module.php(734): MediaWiki\ResourceLoader\Module->buildContent(MediaWiki\ResourceLoader\DerivativeContext)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #13 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/ResourceLoader.php(1204): MediaWiki\ResourceLoader\Module->getModuleContent(MediaWiki\ResourceLoader\DerivativeContext)                                                                           
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #14 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/ResourceLoader.php(1116): MediaWiki\ResourceLoader\ResourceLoader->addOneModuleResponse(MediaWiki\ResourceLoader\DerivativeContext, MediaWiki\ResourceLoader\ReplayMinifierState, string, MediaWiki\ResourceLoader\SiteModule, array)
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #15 /srv/mediawiki/php-1.44.0-wmf.21/includes/ResourceLoader/ResourceLoader.php(1034): MediaWiki\ResourceLoader\ResourceLoader->getOneModuleResponse(MediaWiki\ResourceLoader\DerivativeContext, string, MediaWiki\ResourceLoader\SiteModule)                  
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #16 /srv/mediawiki/php-1.44.0-wmf.21/extensions/WikimediaMaintenance/blameStartupRegistry.php(148): MediaWiki\ResourceLoader\ResourceLoader->makeModuleResponse(MediaWiki\ResourceLoader\DerivativeContext, array)                                             
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #17 /srv/mediawiki/php-1.44.0-wmf.21/maintenance/includes/MaintenanceRunner.php(695): BlameStartupRegistry->execute()
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #18 /srv/mediawiki/php-1.44.0-wmf.21/maintenance/run.php(51): MediaWiki\Maintenance\MaintenanceRunner->run()
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #19 /srv/mediawiki/multiversion/MWScript.php(219): require_once(string)                
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: #20 {main}                                                                             
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: Traceback (most recent call last):                                                     
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]:   File "/usr/local/bin/mw-cli-wrapper", line 61, in <module>                           
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]:     subprocess.run(cmd, check=True, shell=True)                                        
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]:   File "/usr/lib/python3.7/subprocess.py", line 487, in run                            
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]:     output=stdout, stderr=stderr)                                                      
Mar 18 10:30:16 mwmaint2002 mediawiki_job_startupregistrystats-testwiki[18359]: subprocess.CalledProcessError: Command '/usr/local/bin/mwscript extensions/WikimediaMaintenance/blameStartupRegistry.php --wiki testwiki --record-stats' returned non-zero exit status 255.

Notes

Related to T389169?

Details

MediaWiki Version: 1.44.0-wmf.21

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved	Release	jnuche	T386216 1.44.0-wmf.21 deployment blockers
		Resolved	PRODUCTION ERROR	Clement_Goubert	T389182 startupregistrystats-testwiki fails to run on php-1.44.0-wmf.21

Event Timeline

Clement_Goubert created this task.Mar 18 2025, 10:36 AM

Clement_Goubert triaged this task as High priority.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 18 2025, 10:36 AM

Clement_Goubert added a parent task: T386216: 1.44.0-wmf.21 deployment blockers.Mar 18 2025, 10:38 AM

Clement_Goubert added a project: MediaWiki-Platform-Team.Mar 18 2025, 10:41 AM

jnuche raised the priority of this task from High to Unbreak Now!.Mar 18 2025, 10:43 AM

Michael subscribed.Mar 18 2025, 10:48 AM

ihurbain subscribed.Mar 18 2025, 12:33 PM

Live thoughts.

Error coming from this job:

profile::mediawiki::periodic_job { 'startupregistrystats-testwiki':
    command  => '/usr/local/bin/mwscript extensions/WikimediaMaintenance/blameStartupRegistry.php --wiki testwiki --record-stats',

WikimediaMaintenance had no changes this week (list).
"Unsupported operand types" as the error. The line is, appropriately:

			$extraNamespaces = $this->config->get( MainConfigNames::ExtraNamespaces );
…
			// @phan-suppress-next-line PhanTypeMismatchProperty
			$this->namespaceNames = $extraNamespaces +
				$this->localisationCache->getItem( $this->mCode, 'namespaceNames' );

So presumably extraNamespaces isn't an array (or getItem doesn't return one).

Only recent patch to Language.php itself is 9a463504747a5f143e0e81c986a7ccfa5f1c27fd, which went out last week.
Nothing obviously changed about the MainConfigSchema. Did some config change happen in prod?

Is this happening now or are these errors in the past? The recent DB outage (T388646: Fatal DBUnexpectedError: "Database servers in cluster26 are overloaded.") caused a bunch of temporary localization cache issues (e.g. T389001: "LogicException: Process cache for 'en' should be set by now." during session initialization).

Seems fixed now, but the errors I posted were from this morning, so I don't know if it's the temporary cache issues or not. Re-running the script just now worked, so resolving the task.

Krinkle moved this task from Untriaged to Jan-Mar 2025 on the Wikimedia-production-error board.Apr 1 2025, 2:11 AM

DAlangi_WMF mentioned this in T410764: MediaWiki periodic job startupregistrystats-mediawikiwiki failed.Wed, Dec 3, 6:31 PM

startupregistrystats-testwiki fails to run on php-1.44.0-wmf.21Closed, ResolvedPublicPRODUCTION ERRORActions