I am a Release Engineer on the Wikimedia Release-Engineering-Team.
We are definitely running the same version of the code everywhere, it must be something else causing the inconsistent results.
@Mbch331: I'm also getting the same for WikibaseLexeme. That is the current deployed version.
What version do you see on https://www.wikidata.org/wiki/Special:Version
No url in logstash, sadly.
Indeed it is.
AFAIK This is no longer an issue in WMF production. (and iridium doesn't exist anymore)
Yes, and best practice is to use logrotate to cycle your logs periodically in order to avoid running out of space.
Mon, May 21
Sun, May 20
Sat, May 19
Fri, May 18
Swift storage engine now working with tempauth to acquire auth tokens.
@Krinkle: Unfortunately reverting didn't eliminate the errors. Train still blocked.
Thu, May 17
Blame on ApiCSPReport.php isn't very helpful, there doesn't appear to be any recent change to that file. Not sure why this is just now showing up in logs but it's really spammy.
@SBisson: I only reverted this on the branch so we need to do something about it on master or we'll be dealing with it again next week.
@Ciencia_Al_Poder Indeed that is probably caused by this one, however, I just merged the fix so if your problem persists it must be something else.
Wed, May 16
This is broken in production, see T194848: Fatal error: $this is null in Echo/includes/model/Event.php on line 345
@SBisson: Can you take a look at this?
Looks like this was introduced in rECHObe88fc58c1af: Make NotificationJob json-serializable
Tue, May 15
@Zoranzoki21 ok you're added.
ok rEPTW is set to active.
@jcrespo: should we close this then? Have you been able to verify that all are gone?
Mon, May 14
Sun, May 13
@EBernhardson thanks for the update and reminder. That doesn't sound too terribly difficult to deal with although it'll be a bit of work.
Sat, May 12
Fri, May 11
@fgiunchedi: in rPHEX71f853aaf90a: Added code to convert auth credentials into an auth token I've added auth token handling and made the container sharding mandatory based on the first 2 digits of an md5 hash of the object name key.
Thu, May 10
Ok I cherry-picked and deployed the latest patch which seems to have resolved the problem (though I need to monitor for a bit longer to be sure)
@aaron: if you don't think this should be a blocker feel free to remove it.
I'm not sure if this should block the train or not?
Even after deploying rMWfe6fb64d68f4: rdbms: fix LBFactory::commitAll() round handling, this appears to still be occurring.