Page MenuHomePhabricator

8M duplicate parses per day after wmf.23 rollout
Closed, ResolvedPublic

Description

That's the highest we have ever had: https://logstash.wikimedia.org/goto/aba8ee2f1a221abf437d287969313ead

image.png (285×658 px, 18 KB)

Seems to be all from the jobrunners.

Event Timeline

Ladsgroup triaged this task as Unbreak Now! priority.Feb 25 2022, 6:07 PM

Change 766183 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@master] ContentHandler: Use ParserOutputAccess for accessing ParserOutput

https://gerrit.wikimedia.org/r/766183

Ladsgroup lowered the priority of this task from Unbreak Now! to High.Feb 25 2022, 7:24 PM

Given that this is mostly logspam, I will backport this on Monday

Giving a bit a context here, it seems CirrusSearch jobs now do what they used to do in multiple jobs in one now (https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/765577) which is triggering this double parse warnings. They are not technically a new double work but an old one being unearthed.

Change 766183 merged by jenkins-bot:

[mediawiki/core@master] ContentHandler: Use ParserOutputAccess for accessing ParserOutput

https://gerrit.wikimedia.org/r/766183

Change 766136 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[mediawiki/core@wmf/1.38.0-wmf.23] ContentHandler: Use ParserOutputAccess for accessing ParserOutput

https://gerrit.wikimedia.org/r/766136

Change 766136 merged by jenkins-bot:

[mediawiki/core@wmf/1.38.0-wmf.23] ContentHandler: Use ParserOutputAccess for accessing ParserOutput

https://gerrit.wikimedia.org/r/766136

Mentioned in SAL (#wikimedia-operations) [2022-02-28T05:38:39Z] <ladsgroup@deploy1002> Synchronized php-1.38.0-wmf.23/includes/content/ContentHandler.php: Backport: [[gerrit:766136|ContentHandler: Use ParserOutputAccess for accessing ParserOutput (T302620)]] (duration: 00m 49s)

Ladsgroup claimed this task.
Ladsgroup added a project: DBA.

https://logstash.wikimedia.org/goto/83d9236e30d4cda903a31639dc7ab47e
It's back to normalish values:

image.png (205×833 px, 15 KB)

Wikidata ones are still triggering a lot of duplicate parses but I will handle them separately as they are not expensive at all.

On other good news is that this made cirrus search jobs much faster:
https://grafana.wikimedia.org/d/CbmStnlGk/jobqueue-job?orgId=1&var-dc=eqiad%20prometheus%2Fk8s&var-job=cirrusSearchElasticaWrite&from=1646012621506&to=1646033171132

image.png (688×1 px, 133 KB)

Krinkle renamed this task from 8m duplicate parses per day after wmf.23 rollout to 8M duplicate parses per day after wmf.23 rollout.Feb 28 2022, 1:49 PM