Page MenuHomePhabricator

Investigate 1AM enwiki slave lag spikes
Closed, ResolvedPublic

Description

See https://tendril.wikimedia.org/host/view/db1072.eqiad.wmnet/3306 (lag goes from 1-2sec to 20sec).

Tempted to at least partly blame the misc::maintenance::purge_abusefilter job. It runs at 1AM daily and uses 'LIMIT' with UPDATE, which we don't support so the LIMIT is ignored...patch incoming.

Taking that issue to task https://phabricator.wikimedia.org/T95382?workflow=create.

Event Timeline

aaron claimed this task.
aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added a subscriber: aaron.

Change 202657 had a related patch set uploaded (by Aaron Schulz):
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/202657

aaron set Security to None.

Change 202640 had a related patch set uploaded (by Aaron Schulz):
Log huge write queries in CLI scripts

https://gerrit.wikimedia.org/r/202640

Change 202763 had a related patch set uploaded (by Aaron Schulz):
Use wfWaitForSlaves in upload stash cleanup script

https://gerrit.wikimedia.org/r/202763

Change 202792 had a related patch set uploaded (by Aaron Schulz):
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/202792

Change 202640 merged by jenkins-bot:
Log huge write queries in CLI scripts

https://gerrit.wikimedia.org/r/202640

Change 202763 merged by jenkins-bot:
Use wfWaitForSlaves in upload stash cleanup script

https://gerrit.wikimedia.org/r/202763

Change 202657 merged by jenkins-bot:
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/202657

Change 202792 merged by jenkins-bot:
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/202792

Change 203204 had a related patch set uploaded (by Aaron Schulz):
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/203204

Change 203204 merged by jenkins-bot:
Fixed broken batching in –PurgeOldLogIPData

https://gerrit.wikimedia.org/r/203204

Change 203615 had a related patch set uploaded (by Aaron Schulz):
Log huge write queries in CLI scripts

https://gerrit.wikimedia.org/r/203615

Change 203615 merged by Aaron Schulz:
Log huge write queries in CLI scripts

https://gerrit.wikimedia.org/r/203615