See T393963
https://logstash.wikimedia.org/goto/411bbb6dacfa60acf57d4ca579a0a7aa
(Warnings which occur on shutdown have been the majority and they're probably unimportant)
See T393963
https://logstash.wikimedia.org/goto/411bbb6dacfa60acf57d4ca579a0a7aa
(Warnings which occur on shutdown have been the majority and they're probably unimportant)
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | JTweed-WMF | T398814 WE5.1.1 Session storage protection | |||
| Resolved | JTweed-WMF | T400365 Support multiple session stores | |||
| Resolved | matmarex | T362324 Disable PHPSessionHandler in Wikimedia production | |||
| Resolved | matmarex | T400668 Debug warnings that were recorded with $wgPHPSessionHandling = 'warn' in WMF production | |||
| Resolved | matmarex | T402602 Storing objects in session data causes unnecessary session writes, and emits spurious warnings with $wgPHPSessionHandling = 'warn' |
We've logged warnings between 12 May and 11 June, and these entries in Logstash will expire soon, so it's high time to have a look.
There are a few distinct stack traces, but they all ultimately end in one of SessionManager->shutdown(), SessionBackend->resetId() or SessionBackend->unpersist() calling session_write_close(), which in turn calls PHPSessionHandler->write:
Unfortunately this does not tell me much, so I just went to read PHPSessionHandler->write very carefully, and I think I found a possible cause: T402602: Storing objects in session data causes unnecessary session writes, and emits spurious warnings with $wgPHPSessionHandling = 'warn'
After that is fixed, we should try setting $wgPHPSessionHandling = 'warn' again and see if we still get any warnings. I'll also improve the logging, so that they will be more useful this time.
Change #1181001 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/core@master] PHPSessionHandler: In warn mode, report the changed keys
Change #1181001 merged by jenkins-bot:
[mediawiki/core@master] PHPSessionHandler: In warn mode, report the changed keys
Change #1181695 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):
[mediawiki/core@wmf/1.45.0-wmf.15] PHPSessionHandler: In warn mode, report the changed keys
Change #1181695 merged by jenkins-bot:
[mediawiki/core@wmf/1.45.0-wmf.15] PHPSessionHandler: In warn mode, report the changed keys
Mentioned in SAL (#wikimedia-operations) [2025-08-25T13:13:36Z] <lucaswerkmeister-wmde@deploy1003> Started scap sync-world: Backport for [[gerrit:1181695|PHPSessionHandler: In warn mode, report the changed keys (T400668)]], [[gerrit:1181697|Set wgPHPSessionHandling to 'warn' again (T362324)]]
Mentioned in SAL (#wikimedia-operations) [2025-08-25T13:20:04Z] <lucaswerkmeister-wmde@deploy1003> lucaswerkmeister-wmde, matmarex: Backport for [[gerrit:1181695|PHPSessionHandler: In warn mode, report the changed keys (T400668)]], [[gerrit:1181697|Set wgPHPSessionHandling to 'warn' again (T362324)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
Mentioned in SAL (#wikimedia-operations) [2025-08-25T13:26:50Z] <lucaswerkmeister-wmde@deploy1003> Finished scap sync-world: Backport for [[gerrit:1181695|PHPSessionHandler: In warn mode, report the changed keys (T400668)]], [[gerrit:1181697|Set wgPHPSessionHandling to 'warn' again (T362324)]] (duration: 13m 14s)
I did this in the opposite order, so that we can learn more from the logs. All of the improved warnings say Something wrote to $_SESSION['AuthManager::AutoCreateBlacklist']! (https://logstash.wikimedia.org/goto/52a5435c02284314fb7190afb24eb9c4). This is not true (we do not write to $_SESSION), but it does confirm that the problem is with storing objects, since we store StatusValues under the AuthManager::AutoCreateBlacklist key (https://codesearch.wmcloud.org/core/?q=AUTOCREATE_BLOCKLIST). It's also interesting to confirm that this is the only place where storing objects happens (at least in WMF production).