User Details
- User Since
- Jun 27 2020, 12:14 AM (320 w, 3 d)
- Roles
- Administrator
- Availability
- Available
- IRC Nick
- dancy
- LDAP User
- Ahmon Dancy
- MediaWiki User
- ADancy (WMF) [ Global Accounts ]
Thu, Aug 13
Fix deployed in scap 4.280.0
Wed, Aug 12
Tue, Aug 11
Wed, Aug 5
Is there a username/password already prepared for scap to use?
After Release-Engineering-Team discussion today we decided to move forward with https://gerrit.wikimedia.org/r/c/integration/config/+/1184176 with understanding that:
- We'll revert if there are problems.
- There is still no clear process for processing future requests of this type.
This request has my approval.
Tue, Aug 4
@Dzahn Congratulations!
Mon, Aug 3
Should be fixed.
docker-registry.wikimedia.org/repos/releng/buildkit:wmf-v0.32.1@sha256:905218d79b70fa73c6216c5db850c70e1034d2afd1e1c08a52c37fbbb133173b
We assume that attempting to build the localized message cache for all extensions and skins would be too slow at build time and too big at runtime to be viable
update_version/update_version.py comes from https://gerrit.wikimedia.org/g/operations/deployment-charts/+/c6ecb431ad5cf1d5f27ae243739710ed6232e8f8/update_version/update_version.py
Please provide a link to a Jenkins job run.
Fri, Jul 31
debian@deploy:/srv/deployment-charts/helmfile.d/services/eventstreams$ scap deploy-service eventstreamsxz 20:23:41 deploy-service failed: <InvalidDeployServiceConfig> 'eventstreamsxz' is not a service in /etc/scap/service-catalog.yaml (known: eventstreams, shellbox) (scap version: 4.277.0) (duration: \ 00m 00s) ^Gdebian@deploy:/srv/deployment-charts/helmfile.d/services/eventstreams$ scap deploy-service eventstreams 20:23:44 Deploying eventstreams using cluster group wikikube (primary datacenter: eqiad) 20:23:44 [STAGING/DEFAULT] (cd /srv/deployment-charts/helmfile.d/services/eventstreams && helmfile apply -e staging) 20:23:44 [PRODUCTION/CANARY] (cd /srv/deployment-charts/helmfile.d/services/eventstreams && helmfile apply -e codfw -l release=canary) 20:23:44 [PRODUCTION/CANARY] (cd /srv/deployment-charts/helmfile.d/services/eventstreams && helmfile apply -e eqiad -l release=canary) 20:23:44 [PRODUCTION/DEFAULT] (cd /srv/deployment-charts/helmfile.d/services/eventstreams && helmfile apply -e codfw -l release=production) 20:23:44 [PRODUCTION/DEFAULT] (cd /srv/deployment-charts/helmfile.d/services/eventstreams && helmfile apply -e eqiad -l release=production)
Thu, Jul 30
Thanks @colewhite!
Wed, Jul 29
Mention: T429988
@Jdforrester-WMF Please try going back to wmcs runners and let me know how it goes.
An example of an extension.json change that did result in l10n rebuild:
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/cldr/+/1318704
https://gerrit.wikimedia.org/r/c/operations/puppet/+/1308767 (buildkitd: Bump buildkit image to wmf-v0.31.1) was merged Jul 08 20:34 UTC.
The change is deployed.
Tue, Jul 28
Hello! I'm on train duty this week and this warning tends to be the second most frequent message in the mediawiki error logs when I periodically check on them. It's not causing a big problem but it would be nice to eliminate.
Please note T433429
Mon, Jul 27
Fri, Jul 24
Here's what's happening today:
dancy@deploy1003:~$ scap mwscript -- eval.php --wiki testcommonswiki
Warning: socket_sendto(): Unable to write to socket [101]: Network is unreachable in /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/Logger/Monolog/LegacyHandler.php on line 224
[95471c8b66f3baec2522299a] [no req] UnexpectedValueException: Section 's4' has no hosts defined.
Backtrace:
from /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/libs/Rdbms/LBFactory/LBFactoryMulti.php(158)
#0 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/libs/Rdbms/LBFactory/LBFactoryMulti.php(208): Wikimedia\Rdbms\LBFactoryMulti->newMainLB(bool)
#1 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/Storage/NameTableStoreFactory.php(85): Wikimedia\Rdbms\LBFactoryMulti->getMainLB(bool)
#2 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/Storage/NameTableStoreFactory.php(106): MediaWiki\Storage\NameTableStoreFactory->get(string, bool)
#3 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/ServiceWiring.php(605): MediaWiki\Storage\NameTableStoreFactory->getChangeTagDef()
#4 /srv/mediawiki-staging/php-1.47.0-wmf.12/vendor/wikimedia/services/src/ServiceContainer.php(440): Wikimedia\Services\ServiceContainer::{closure}(MediaWiki\MediaWikiServices)
#5 /srv/mediawiki-staging/php-1.47.0-wmf.12/vendor/wikimedia/services/src/ServiceContainer.php(406): Wikimedia\Services\ServiceContainer->createService(string)
#6 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/MediaWikiServices.php(392): Wikimedia\Services\ServiceContainer->getService(string)
#7 /srv/mediawiki-staging/php-1.47.0-wmf.12/vendor/wikimedia/services/src/ServiceContainer.php(414): MediaWiki\MediaWikiServices->getService(string)
#8 /srv/mediawiki-staging/php-1.47.0-wmf.12/vendor/wikimedia/object-factory/src/ObjectFactory.php(184): Wikimedia\Services\ServiceContainer->get(string)
#9 /srv/mediawiki-staging/php-1.47.0-wmf.12/vendor/wikimedia/object-factory/src/ObjectFactory.php(130): Wikimedia\ObjectFactory\ObjectFactory::getObjectFromSpec(array, array)
#10 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/HookContainer/HookContainer.php(223): Wikimedia\ObjectFactory\ObjectFactory->createObject(array)
#11 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/HookContainer/HookContainer.php(280): MediaWiki\HookContainer\HookContainer->makeExtensionHandlerCallback(string, array, array)
#12 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/HookContainer/HookContainer.php(403): MediaWiki\HookContainer\HookContainer->normalizeHandler(string, array, array)
#13 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/HookContainer/HookContainer.php(120): MediaWiki\HookContainer\HookContainer->getHandlers(string, array)
#14 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/HookContainer/HookRunner.php(3938): MediaWiki\HookContainer\HookContainer->run(string, array)
#15 /srv/mediawiki-staging/php-1.47.0-wmf.12/includes/Setup.php(476): MediaWiki\HookContainer\HookRunner->onSetupAfterCache()
#16 /srv/mediawiki-staging/php-1.47.0-wmf.12/maintenance/run.php(52): require_once(string)
#17 /srv/mediawiki-staging/multiversion/MWScript.php(219): require_once(string)
#18 {main}Thanks @Krinkle!
Thu, Jul 23
For the record, none of the entries in the NOT PRESENT list have a corresponding configuration in https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/dockerfiles/, so everything in that list has at some time in the past been considered "too old" by someone maintaining the dockerfiles. My opinion is that they can safely be excluded.
Thanks @Krinkle. I'm maxing out the priority now that I've rolled the train to group2 and the error rate is quite high now.
Thanks to everyone who helped on this!
Another deletedProperty.cy.ts test failure: https://integration.wikimedia.org/ci/job/quibble-with-Wikibase-extensions-browser-tests-only-vendor-php83/15705/console
Wed, Jul 22
This problem happened again today. The conclusion in today's case is that the react-is package was republished (verified by timestamp) while three jobs were simultaneously running npm ci, all starting around the same time. They all started out with the same copy of the cache, loaded from castor. Some of them downloaded one version of the react-is file and some of them got another version. The file is stored in content-addressable directory structure, which ultimately just means that the filename is based on the hash of the content. In addition there is an index which effectively maps a package name to its hashed filename. Although hashed filename differs based on the content, the length of the filename is always the same (whatever the size of the hash is). This sets up a situation where the index file across the builds may have different content (due to different hashes for the react-is package), but the same size and mtime (if the changes are made within the same second across jobs, which did happen).
@Krinkle Bringing this to your attention
Possibly related:https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1309703
- mwversion: 1.47.0-wmf.12
- timestamp: 2026-07-22T19:02:43.747Z
- phpversion: 8.3.32
- reqId: 518c2f6f-fa12-4b56-bc2e-0f5af527dbe4
- Find reqId in Logstash
Tue, Jul 21
Possibly related to https://gerrit.wikimedia.org/r/c/mediawiki/extensions/AutoModerator/+/1310686
Claude's analysis:
Mon, Jul 20
Jul 17 2026
@colewhite Can you give us some insight into what's going with logging-logstash-04.logging.eqiad1.wikimedia.cloud? It has failed twice this week.
Jul 16 2026
Jul 14 2026
@bd808 Seems like this ticket may be ready to close?
Jul 13 2026
Fix deployed in scap 4.274.0.
I retested today and the problem no longer exists, so I'm satisfied.
Adding an associated error for searchability:
@Scott_French Scap 4.273.0 has been deployed with this change. mediawiki-deployments.yaml is ready to be extended.
There are still ~1000 hits for this message in the last 15 minutes.
Jul 9 2026
Jul 8 2026
I used this program:
import java.io.*; import java.security.*; import java.util.Base64;