Page MenuHomePhabricator

Fatal exception of type "BadMethodCallException" on MediaWiki.org Special:RecentChanges filter widget
Closed, ResolvedPublicPRODUCTION ERROR

Description

Error
  • mwversion: 1.38.0-wmf.23
  • reqId: 77ac85ad-b115-4a0e-8692-91659ad58ace
normalized_message
Generating module package failed: BadMethodCallException: Sessions are disabled for load entry point
exception.trace
from /srv/mediawiki/php-1.38.0-wmf.23/includes/session/SessionManager.php(893)
#0 /srv/mediawiki/php-1.38.0-wmf.23/includes/session/SessionManager.php(258): MediaWiki\Session\SessionManager->getSessionFromInfo(MediaWiki\Session\SessionInfo, WebRequest)
#1 /srv/mediawiki/php-1.38.0-wmf.23/includes/WebRequest.php(830): MediaWiki\Session\SessionManager->getSessionForRequest(WebRequest)
#2 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/User.php(1111): WebRequest->getSession()
#3 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/User.php(435): User->loadFromSession()
#4 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/User.php(1893): User->load()
#5 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/User.php(2531): User->getId()
#6 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/UserOptionsManager.php(644): User->isRegistered()
#7 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/UserOptionsManager.php(496): MediaWiki\User\UserOptionsManager->getCacheKey(User)
#8 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/UserOptionsManager.php(147): MediaWiki\User\UserOptionsManager->loadUserOptions(User, integer)
#9 /srv/mediawiki/php-1.38.0-wmf.23/includes/user/User.php(2357): MediaWiki\User\UserOptionsManager->getOption(User, string, NULL, boolean)
#10 /srv/mediawiki/php-1.38.0-wmf.23/includes/context/RequestContext.php(384): User->getOption(string)
#11 /srv/mediawiki/php-1.38.0-wmf.23/includes/StubUserLang.php(34): RequestContext->getLanguage()
#12 /srv/mediawiki/php-1.38.0-wmf.23/includes/StubObject.php(223): StubUserLang->_newObject()
#13 /srv/mediawiki/php-1.38.0-wmf.23/includes/StubObject.php(103): StubObject->_unstub(string, integer)
#14 /srv/mediawiki/php-1.38.0-wmf.23/includes/content/ContentHandler.php(737): StubObject::unstub(StubUserLang)
#15 /srv/mediawiki/php-1.38.0-wmf.23/includes/Title.php(3972): ContentHandler->getPageLanguage(Title)
#16 /srv/mediawiki/php-1.38.0-wmf.23/extensions/Translate/messagegroups/WikiPageMessageGroup.php(37): Title->getPageLanguage()
#17 /srv/mediawiki/php-1.38.0-wmf.23/extensions/Translate/messagegroups/MessageGroupOld.php(277): WikiPageMessageGroup->getSourceLanguage()
#18 /srv/mediawiki/php-1.38.0-wmf.23/extensions/Translate/messagegroups/WikiPageMessageGroup.php(140): MessageGroupOld->isSourceLanguage(string)
#19 /srv/mediawiki/php-1.38.0-wmf.23/extensions/Translate/tag/TranslatablePage.php(241): WikiPageMessageGroup->getMessage(string, string, integer)
#20 /srv/mediawiki/php-1.38.0-wmf.23/extensions/Translate/tag/PageTranslationHooks.php(86): TranslatablePage->getPageDisplayTitle(string)
#21 /srv/mediawiki/php-1.38.0-wmf.23/includes/HookContainer/HookContainer.php(338): PageTranslationHooks::renderTagPage(Parser, string, StripState)
#22 /srv/mediawiki/php-1.38.0-wmf.23/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#23 /srv/mediawiki/php-1.38.0-wmf.23/includes/HookContainer/HookRunner.php(2835): MediaWiki\HookContainer\HookContainer->run(string, array)
#24 /srv/mediawiki/php-1.38.0-wmf.23/includes/parser/Parser.php(1564): MediaWiki\HookContainer\HookRunner->onParserBeforeInternalParse(Parser, string, StripState)
#25 /srv/mediawiki/php-1.38.0-wmf.23/includes/parser/Parser.php(699): Parser->internalParse(string)
#26 /srv/mediawiki/php-1.38.0-wmf.23/includes/cache/MessageCache.php(1322): Parser->parse(string, MediaWiki\Page\PageReferenceValue, ParserOptions, boolean)
#27 /srv/mediawiki/php-1.38.0-wmf.23/includes/language/Message.php(1433): MessageCache->parse(string, MediaWiki\Page\PageReferenceValue, boolean, boolean, LanguageEn)
#28 /srv/mediawiki/php-1.38.0-wmf.23/includes/language/Message.php(990): Message->parseText(string)
#29 /srv/mediawiki/php-1.38.0-wmf.23/includes/language/Message.php(1048): Message->format(string)
#30 /srv/mediawiki/php-1.38.0-wmf.23/includes/changetags/ChangeTags.php(1848): Message->parse()
#31 /srv/mediawiki/php-1.38.0-wmf.23/includes/specialpage/ChangesListSpecialPage.php(871): ChangeTags::getChangeTagList(ResourceLoaderContext, LanguageEn)
#32 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoaderFileModule.php(1390): ChangesListSpecialPage::getRcFiltersConfigVars(ResourceLoaderContext, GlobalVarConfig, NULL)
#33 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoaderFileModule.php(365): ResourceLoaderFileModule->getPackageFiles(ResourceLoaderContext)
#34 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoaderModule.php(796): ResourceLoaderFileModule->getScript(ResourceLoaderContext)
#35 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoaderModule.php(765): ResourceLoaderModule->buildContent(ResourceLoaderContext)
#36 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoader.php(1148): ResourceLoaderModule->getModuleContent(ResourceLoaderContext)
#37 /srv/mediawiki/php-1.38.0-wmf.23/includes/resourceloader/ResourceLoader.php(850): ResourceLoader->makeModuleResponse(ResourceLoaderContext, array, array)
#38 /srv/mediawiki/php-1.38.0-wmf.23/load.php(52): ResourceLoader->respond(ResourceLoaderContext)
#39 /srv/mediawiki/php-1.38.0-wmf.23/load.php(38): wfLoadMain()
#40 /srv/mediawiki/w/load.php(3): require(string)
#41 {main}
Impact

Filter widget on Special:RecentChanges doesn't load

Notes

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Zabe set Phatality ID (Deprecated) to b92ab35a88f4f126c40f111ee15270dbf0c7006bcabaaf87347dc7c2b28e2315.

These started happening around 2022-02-25. There have been 596 instances since then.

kostajh moved this task from Needs Discussion to Triaged on the Growth-Team board.
kostajh subscribed.

I think this is something the MediaWiki-extensions-Translate maintainers (Language-Team) would be best equipped to deal with. From what I can see, the problem originates in PageTranslationHooks::renderTagPage; admittedly, this code hasn't changed much in many years but it doesn't look like the source of the problem is in any RecentChanges code.

Nikerabbit subscribed.
  1. In https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+blame/heads/wmf/1.38.0-wmf.23/includes/changetags/ChangeTags.php#1848 there is a call for Messsage::parse for message.
  2. It's not easy to say which message this is, but the patterns are tag-* and tag-*-description.
  3. My initial guess is that someone on MediaWiki.org has included a translatable page
  4. Anyway, Translate extension is invoked to process a page with <translate> tags
  5. To render the page, it needs to fetch the display title from translations
  6. WikiPageMessageGroup wants to know if the language asked is the source language or not
  7. ContentHandler::getPageLanguage explicitly unstubs $wgLang, which is not allowed for load.php requests

My conclusion: It is not safe to call any code that uses Message::parse from load.php requests.

As for what messages exactly cause the problem, on mediawiki.org there are bunch of messages like:

{{Tag description}} is a template that has been marked for translation in 2020-05: https://www.mediawiki.org/w/index.php?title=Template:Tag_description&oldid=3824487

None of the above explains why it only started happening more recently.

I see that T27909: Add a drop-down list for the tags in Special:Newpages, Special:Log and Special:Contributions went out with that train, but it doesn't seem to touch any code paths in the trace.

To summarize: I have no idea which exact message is failing, and I have no idea what change may have triggered this. I don't see how this could be fixed in Translate, as we need to call Title::getPageLanguage and it is unexpected to us that this triggers a sequence of unstubbing wgLang which attempts to load user settings which attempts to access a session.

I think your best option would be to treat T299369: Consider removing global $userLang from onPageContentLanguage hook as a blocker and request help from platform engineering.

Still seen in production error logs.

[…] I have no idea which exact message is failing, and […]

Trying to narrow it down, I see it affects mutiple skins so the recent rollout of vector-2022 has probably not caused or amplified this. It also affects multiple languages as well, so it is not limited to English, although only a small number of other languages are affected, and do they fail in a subtly different way depending on the language. This suggests to me that it is indeed caused and dependent on the specific content of a message.

For mediawiki.org modules=rcfilters vector=vector lang=fr, the logs show the following:

#25 /srv/mediawiki/php-1.38.0-wmf.25/includes/parser/Parser.php(699): Parser->internalParse('[[{{MediaWiki:v...')
#26 /srv/mediawiki/php-1.38.0-wmf.25/includes/cache/MessageCache.php(1323): Parser->parse('[[{{MediaWiki:v...', Object(MediaWiki\Page\PageReferenceValue), Object(ParserOptions), true)
#27 /srv/mediawiki/php-1.38.0-wmf.25/includes/language/Message.php(1433): MessageCache->parse('[[{{MediaWiki:v...', 
#28 /srv/mediawiki/php-1.38.0-wmf.25/includes/language/Message.php(990): Message->parseText('[[{{MediaWiki:v...')
#29 /srv/mediawiki/php-1.38.0-wmf.25/includes/language/Message.php(1048): Message->format('parse')
#30 /srv/mediawiki/php-1.38.0-wmf.25/includes/changetags/ChangeTags.php(1853): Message->parse()
#31 /srv/mediawiki/php-1.38.0-wmf.25/includes/specialpage/ChangesListSpecialPage.php(870): ChangeTags::getChangeTagList

Codesearch: "\[\{\{MediaWiki:v" with file:fr.json and repos:deployed

VisualEditor / i18n/ve-wmf/fr.json
42	"tag-visualeditor": "[[{{MediaWiki:visualeditor-descriptionpagelink}}|Éditeur visuel]]",

I also confirmed that it has no overrides on mediawiki.org, has not changed in many years, and the same for MediaWiki:visualeditor-descriptionpagelink. So still unsure why it changed recently.

Inspired by @Krinkle I also went browsing the logs, and found this one very interesting: https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-mediawiki-2022.03.13?id=I4XihH8BSK2dFIgQNXtq

Apparently we are just trying to parse the wikitext Mobile edit (from https://meta.wikimedia.org/wiki/MediaWiki:Tag-mobile_edit), and somehow Translate gets involved and throws an exception. This might make it easier to narrow down the cause?

(I also briefly wondered whether my own recent-ish changes to tags code (1) (2) (3) (4) could be responsible, but I don't see anything in them that could cause this, and the timing also doesn't match. It wouldn't hurt if someone else reviewed them more carefully, though.)

It should be short-circuited at Translate/tag/PageTranslationHooks.php#66, but the expected title for message parsing in load.php (Special:Badtitle/ResourceLoaderContext) somehow is overridden and passed all the checks?

Change 771328 had a related patch set uploaded (by Nikerabbit; author: Nikerabbit):

[mediawiki/extensions/Translate@master] Add debug logging to investigate T302754

https://gerrit.wikimedia.org/r/771328

Another oddity I noticed, but it shouldn't case this issue:

(metawiki)> select * from revtag where rt_page = 0;
+---------+---------+-------------+----------+
| rt_type | rt_page | rt_revision | rt_value |
+---------+---------+-------------+----------+
|         |       0 |           0 | NULL     |
+---------+---------+-------------+----------+

Change 771328 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] Add debug logging to investigate T302754

https://gerrit.wikimedia.org/r/771328

hashar subscribed.

That is still occurring (T304931).

Yesterday, https://www.mediawiki.org/wiki/Special:RecentChanges?uselang=es (note user language = es) was working fine. Today, the filter area is in a permanent "loading" state, as well as Special:Watchlist.

Impact:

Special:RecentChanges and Special:Watchlist of users with Spanish (es) interface language. English doesn't seem to be affected. I haven't identified other languages with that problem (I checked like 3 or 4).

Possible big impact on Spanish WMF wikis (I don't know if this affects other wikis outside of MediaWiki.org)

from /srv/mediawiki/php-1.39.0-wmf.5/includes/session/SessionManager.php(893)
#0 /srv/mediawiki/php-1.39.0-wmf.5/includes/session/SessionManager.php(372): MediaWiki\Session\SessionManager->getSessionFromInfo(MediaWiki\Session\SessionInfo, WebRequest)
#1 /srv/mediawiki/php-1.39.0-wmf.5/includes/session/SessionManager.php(306): MediaWiki\Session\SessionManager->getEmptySessionInternal(WebRequest)
#2 /srv/mediawiki/php-1.39.0-wmf.5/includes/session/SessionManager.php(256): MediaWiki\Session\SessionManager->getEmptySession(WebRequest)
#3 /srv/mediawiki/php-1.39.0-wmf.5/includes/WebRequest.php(833): MediaWiki\Session\SessionManager->getSessionForRequest(WebRequest)
#4 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/User.php(1111): WebRequest->getSession()
#5 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/User.php(435): User->loadFromSession()
#6 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/User.php(1896): User->load()
#7 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/User.php(2534): User->getId()
#8 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/UserOptionsManager.php(644): User->isRegistered()
#9 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/UserOptionsManager.php(496): MediaWiki\User\UserOptionsManager->getCacheKey(User)
#10 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/UserOptionsManager.php(147): MediaWiki\User\UserOptionsManager->loadUserOptions(User, integer)
#11 /srv/mediawiki/php-1.39.0-wmf.5/includes/user/User.php(2360): MediaWiki\User\UserOptionsManager->getOption(User, string, NULL, boolean)
#12 /srv/mediawiki/php-1.39.0-wmf.5/includes/context/RequestContext.php(450): User->getOption(string)
#13 /srv/mediawiki/php-1.39.0-wmf.5/includes/StubUserLang.php(34): RequestContext->getLanguage()
#14 /srv/mediawiki/php-1.39.0-wmf.5/includes/StubObject.php(223): StubUserLang->_newObject()
#15 /srv/mediawiki/php-1.39.0-wmf.5/includes/StubObject.php(103): StubObject->_unstub(string, integer)
#16 /srv/mediawiki/php-1.39.0-wmf.5/includes/content/ContentHandler.php(737): StubObject::unstub(StubUserLang)
#17 /srv/mediawiki/php-1.39.0-wmf.5/includes/Title.php(3951): ContentHandler->getPageLanguage(Title)
#18 /srv/mediawiki/php-1.39.0-wmf.5/extensions/Translate/messagegroups/WikiPageMessageGroup.php(37): Title->getPageLanguage()
#19 /srv/mediawiki/php-1.39.0-wmf.5/extensions/Translate/messagegroups/MessageGroupOld.php(277): WikiPageMessageGroup->getSourceLanguage()
#20 /srv/mediawiki/php-1.39.0-wmf.5/extensions/Translate/messagegroups/WikiPageMessageGroup.php(140): MessageGroupOld->isSourceLanguage(string)
#21 /srv/mediawiki/php-1.39.0-wmf.5/extensions/Translate/tag/TranslatablePage.php(236): WikiPageMessageGroup->getMessage(string, string, integer)
#22 /srv/mediawiki/php-1.39.0-wmf.5/extensions/Translate/tag/PageTranslationHooks.php(73): TranslatablePage->getPageDisplayTitle(string)
#23 /srv/mediawiki/php-1.39.0-wmf.5/includes/HookContainer/HookContainer.php(338): PageTranslationHooks::renderTagPage(Parser, string, StripState)
#24 /srv/mediawiki/php-1.39.0-wmf.5/includes/HookContainer/HookContainer.php(137): MediaWiki\HookContainer\HookContainer->callLegacyHook(string, array, array, array)
#25 /srv/mediawiki/php-1.39.0-wmf.5/includes/HookContainer/HookRunner.php(2835): MediaWiki\HookContainer\HookContainer->run(string, array)
#26 /srv/mediawiki/php-1.39.0-wmf.5/includes/parser/Parser.php(1565): MediaWiki\HookContainer\HookRunner->onParserBeforeInternalParse(Parser, string, StripState)
#27 /srv/mediawiki/php-1.39.0-wmf.5/includes/parser/Parser.php(700): Parser->internalParse(string)
#28 /srv/mediawiki/php-1.39.0-wmf.5/includes/cache/MessageCache.php(1325): Parser->parse(string, MediaWiki\Page\PageReferenceValue, ParserOptions, boolean)
#29 /srv/mediawiki/php-1.39.0-wmf.5/includes/language/Message.php(1445): MessageCache->parse(string, MediaWiki\Page\PageReferenceValue, boolean, boolean, Language)
#30 /srv/mediawiki/php-1.39.0-wmf.5/includes/language/Message.php(1002): Message->parseText(string)
#31 /srv/mediawiki/php-1.39.0-wmf.5/includes/language/Message.php(1060): Message->format(string)
#32 /srv/mediawiki/php-1.39.0-wmf.5/includes/changetags/ChangeTags.php(1853): Message->parse()
#33 /srv/mediawiki/php-1.39.0-wmf.5/includes/specialpage/ChangesListSpecialPage.php(870): ChangeTags::getChangeTagList(ResourceLoaderContext, Language)
#34 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoaderFileModule.php(1390): ChangesListSpecialPage::getRcFiltersConfigVars(ResourceLoaderContext, GlobalVarConfig, NULL)
#35 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoaderFileModule.php(365): ResourceLoaderFileModule->getPackageFiles(ResourceLoaderContext)
#36 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoaderModule.php(814): ResourceLoaderFileModule->getScript(ResourceLoaderContext)
#37 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoaderModule.php(783): ResourceLoaderModule->buildContent(ResourceLoaderContext)
#38 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoader.php(1148): ResourceLoaderModule->getModuleContent(ResourceLoaderContext)
#39 /srv/mediawiki/php-1.39.0-wmf.5/includes/resourceloader/ResourceLoader.php(850): ResourceLoader->makeModuleResponse(ResourceLoaderContext, array, array)
#40 /srv/mediawiki/php-1.39.0-wmf.5/load.php(52): ResourceLoader->respond(ResourceLoaderContext)
#41 /srv/mediawiki/php-1.39.0-wmf.5/load.php(38): wfLoadMain()
#42 /srv/mediawiki/w/load.php(3): require(string)
#43 {main}

@Ciencia_Al_Poder indicated it might be triggered by an i18n message.

The event log has:

messageT302754 Failed to set display title for page Reporting_security_bugs/es
pageid1523346
text[[{{MediaWiki:visualeditor-descriptionpagelink}}|Edición visual]]
urlhttps://www.mediawiki.org/w/load.php?lang=es&modules=mediawiki.rcfilters.filters.ui&skin=vector-2022&version=19eng

That page is not a translatable page. I'll investigate some more.

That page is not a translatable page. I'll investigate some more.

Well, it's also possible to have a translatable page involved, I have left a comment on change 757974.

Simply add var_dump($text); in PageTranslationHooks::renderTagPage() and visit history page of a translated page (language subpage),
then you can see:

string(353) "
Diff selection: Mark the radio buttons of the revisions to compare and hit enter or the button at the bottom.
Legend: ({{int:cur}}) = difference with latest revision, ({{int:last}}) = difference with preceding revision, {{int:minoreditletter}} = minor edit.
" string(21) "Retrieved from "$'"1"" string(23) "Change language variant"

And it also attempted to set the display title, but it was overridden by the correct title ([Title]: Revision history) afterward.
It only breaks during load.php requests, because the OutputPage is not available.

Change 774899 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/Translate@master] PageTranslationHooks: Don't kick in during message parsing

https://gerrit.wikimedia.org/r/774899

Change 775015 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775015

Change 775015 merged by jenkins-bot:

[mediawiki/core@master] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775015

Thanks for the investigation and the fix @Func. I'll backport it today (https://wikitech.wikimedia.org/wiki/Deployments#deploycal-item-20220331T1300) and hopefully that will resolve the issue.

Summary of the problem, as I understand it: We were caching Message objects in MainWANObjectCache. They can have a page title associated with them, used as the context title when parsing wikitext. When we fetched that data, we ended up with those completely unrelated titles, which broke assumptions in Translate extension code. The solution is to only cache strings, and rebuild the Message objects from scratch when needed.

Change 775437 had a related patch set uploaded (by Bartosz Dziewoński; author: Func):

[mediawiki/core@wmf/1.39.0-wmf.5] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775437

Change 775438 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@REL1_38] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775438

The error has been happening intermittently, and I can't reproduce it right now, so I guess we should leave this open for a few days to confirm that it no longer occurs.

Logstash query to watch: https://logstash.wikimedia.org/goto/60858c31c103268d3d0f7d147b7068c4
Current results:

image.png (555×2 px, 49 KB)

Change 775437 merged by jenkins-bot:

[mediawiki/core@wmf/1.39.0-wmf.5] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775437

Mentioned in SAL (#wikimedia-operations) [2022-03-31T13:55:56Z] <lucaswerkmeister-wmde@deploy1002> Synchronized php-1.39.0-wmf.5/includes/changetags/ChangeTags.php: Backport: [[gerrit:775437|ChangeTags: Use localizer with correct page title to parse messages (T302754)]] (duration: 00m 51s)

Change 774899 merged by jenkins-bot:

[mediawiki/extensions/Translate@master] PageTranslationHooks: Don't kick in during interface message parsing

https://gerrit.wikimedia.org/r/774899

Change 775438 merged by jenkins-bot:

[mediawiki/core@REL1_38] ChangeTags: Use localizer with correct page title to parse messages

https://gerrit.wikimedia.org/r/775438

The error has been happening intermittently, and I can't reproduce it right now, so I guess we should leave this open for a few days to confirm that it no longer occurs.

Logstash query to watch: https://logstash.wikimedia.org/goto/60858c31c103268d3d0f7d147b7068c4
Current results:

image.png (555×2 px, 49 KB)

Now:

image.png (555×2 px, 45 KB)

No errors in the last 3 days, seems fixed for sure.

@matmarex I see this error on beta cluster. For example, just going to https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page:

[ZDlKnxdXJNPNmiKj8AWTdgAAABE] /w/load.php?lang=en&modules=startup&only=scripts&raw=1&skin=vector-2022   BadMethodCallException: Sessions are disabled for load entry point
Backtrace:
from /srv/mediawiki/php-master/includes/session/SessionManager.php(900)
#0 /srv/mediawiki/php-master/includes/session/SessionManager.php(249): MediaWiki\Session\SessionManager->getSessionFromInfo(MediaWiki\Session\SessionInfo, WebRequest)
#1 /srv/mediawiki/php-master/includes/WebRequest.php(840): MediaWiki\Session\SessionManager->getSessionForRequest(WebRequest)
#2 /srv/mediawiki/php-master/includes/user/User.php(1105): WebRequest->getSession()
#3 /srv/mediawiki/php-master/includes/user/User.php(443): User->loadFromSession()
#4 /srv/mediawiki/php-master/includes/user/User.php(1661): User->load()
#5 /srv/mediawiki/php-master/includes/user/User.php(2313): User->getId()
#6 /srv/mediawiki/php-master/includes/user/UserOptionsManager.php(650): User->isRegistered()
#7 /srv/mediawiki/php-master/includes/user/UserOptionsManager.php(502): MediaWiki\User\UserOptionsManager->getCacheKey(User)
#8 /srv/mediawiki/php-master/includes/user/UserOptionsManager.php(150): MediaWiki\User\UserOptionsManager->loadUserOptions(User, integer)
#9 /srv/mediawiki/php-master/includes/context/RequestContext.php(443): MediaWiki\User\UserOptionsManager->getOption(User, string)
#10 /srv/mediawiki/php-master/includes/StubObject/StubUserLang.php(40): RequestContext->getLanguage()
#11 /srv/mediawiki/php-master/includes/StubObject/StubObject.php(232): MediaWiki\StubObject\StubUserLang->_newObject()
#12 /srv/mediawiki/php-master/includes/StubObject/StubObject.php(124): MediaWiki\StubObject\StubObject->_unstub(string, integer)
#13 /srv/mediawiki/php-master/includes/StubObject/StubObject.php(155): MediaWiki\StubObject\StubObject->_call(string, array)
#14 /srv/mediawiki/php-master/includes/language/LanguageConverterFactory.php(181): MediaWiki\StubObject\StubObject->__call(string, array)
#15 /srv/mediawiki/php-master/includes/title/Title.php(219): MediaWiki\Languages\LanguageConverterFactory->getLanguageConverter(MediaWiki\StubObject\StubUserLang)
#16 /srv/mediawiki/php-master/includes/title/Title.php(4039): MediaWiki\Title\Title->getLanguageConverter(MediaWiki\StubObject\StubUserLang)
#17 /srv/mediawiki/php-master/includes/parser/Parser.php(5492): MediaWiki\Title\Title->getPageViewLanguage()
#18 /srv/mediawiki/php-master/includes/parser/Parser.php(2694): Parser->makeImage(MediaWiki\Title\Title, array, LinkHolderArray)
#19 /srv/mediawiki/php-master/includes/parser/Parser.php(2437): Parser->handleInternalLinks2(string)
#20 /srv/mediawiki/php-master/includes/parser/Parser.php(1616): Parser->handleInternalLinks(string)
#21 /srv/mediawiki/php-master/includes/parser/Parser.php(700): Parser->internalParse(string)
#22 /srv/mediawiki/php-master/includes/language/MessageCache.php(1481): Parser->parse(string, MediaWiki\Page\PageReferenceValue, ParserOptions, boolean)
#23 /srv/mediawiki/php-master/includes/language/Message.php(1428): MessageCache->parse(string, MediaWiki\Page\PageReferenceValue, boolean, boolean, LanguageEn)
#24 /srv/mediawiki/php-master/includes/language/Message.php(1005): Message->parseText(string)
#25 /srv/mediawiki/php-master/includes/language/Message.php(1043): Message->format(string)
#26 /srv/mediawiki/php-master/extensions/VisualEditor/includes/VisualEditorDataModule.php(32): Message->parse()
#27 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(816): MediaWiki\Extension\VisualEditor\VisualEditorDataModule->getScript(MediaWiki\ResourceLoader\Context)
#28 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(785): MediaWiki\ResourceLoader\Module->buildContent(MediaWiki\ResourceLoader\Context)
#29 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(925): MediaWiki\ResourceLoader\Module->getModuleContent(MediaWiki\ResourceLoader\Context)
#30 /srv/mediawiki/php-master/includes/ResourceLoader/StartUpModule.php(223): MediaWiki\ResourceLoader\Module->getVersionHash(MediaWiki\ResourceLoader\Context)
#31 /srv/mediawiki/php-master/includes/ResourceLoader/StartUpModule.php(412): MediaWiki\ResourceLoader\StartUpModule->getModuleRegistrations(MediaWiki\ResourceLoader\Context)
#32 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(816): MediaWiki\ResourceLoader\StartUpModule->getScript(MediaWiki\ResourceLoader\Context)
#33 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(785): MediaWiki\ResourceLoader\Module->buildContent(MediaWiki\ResourceLoader\Context)
#34 /srv/mediawiki/php-master/includes/ResourceLoader/Module.php(925): MediaWiki\ResourceLoader\Module->getModuleContent(MediaWiki\ResourceLoader\Context)
#35 /srv/mediawiki/php-master/includes/ResourceLoader/ResourceLoader.php(688): MediaWiki\ResourceLoader\Module->getVersionHash(MediaWiki\ResourceLoader\Context)
#36 [internal function]: MediaWiki\ResourceLoader\ResourceLoader->MediaWiki\ResourceLoader\{closure}(string)
#37 /srv/mediawiki/php-master/includes/ResourceLoader/ResourceLoader.php(686): array_map(Closure, array)
#38 /srv/mediawiki/php-master/includes/ResourceLoader/ResourceLoader.php(787): MediaWiki\ResourceLoader\ResourceLoader->getCombinedVersion(MediaWiki\ResourceLoader\Context, array)
#39 /srv/mediawiki/php-master/load.php(53): MediaWiki\ResourceLoader\ResourceLoader->respond(MediaWiki\ResourceLoader\Context)
#40 /srv/mediawiki/php-master/load.php(39): wfLoadMain()
#41 /srv/mediawiki/w/load.php(3): require(string)
#42 {main}

@matmarex I see this error on beta cluster. For example, just going to https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page:

This one is caused by commit 42aa5f9 recently and should be filed separately.