Page MenuHomePhabricator

Find and fix duplicate CDN purges from MediaWiki
Closed, ResolvedPublic

Description

Some updates going through SquidUpdate have been sending some URL purges twice per request for years. According to bblack, this is an ongoing problem.

The CRUD cycle for pages and files should be audited. I'll also have too look at what extension hooks are doing too.

It might also make sense to have a generic method that batches all purges for a request and de-duplicates/runs them at the end. Each separate purge() call already de-duplicates now.

Details

Related Gerrit Patches:

Event Timeline

aaron created this task.Nov 19 2015, 12:05 AM
aaron claimed this task.
aaron raised the priority of this task from to Needs Triage.
aaron updated the task description. (Show Details)
aaron added a project: Performance-Team.
aaron added a subscriber: aaron.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 19 2015, 12:05 AM
aaron moved this task from Inbox to Doing on the Performance-Team board.Nov 23 2015, 7:52 PM
aaron added a subscriber: BBlack.Nov 25 2015, 12:24 AM

@BBlack : which of articles, File: pages, uploads, and thumbnails does this occur for? Is it all of them? I also wonder if this happens on every edit or just some portion of them.

Rather than just hunting them down, I wonder if it would be useful to have a interface for accumulating purges and issuing them all near the end of the request (which would better de-duplicate them).

Change 256144 had a related patch set uploaded (by Aaron Schulz):
Make CDN purge calls use DeferredUpdates

https://gerrit.wikimedia.org/r/256144

@aaron - I haven't had time to investigate this, but in light of T119038 (issues with upload cache purges currently - probably related to multicast changes), it's probably best to avoid purge behavior changes until that's resolved.

Denniss added a subscriber: Denniss.Dec 1 2015, 1:01 AM

Change 256144 merged by jenkins-bot:
Make CDN purge calls use DeferredUpdates

https://gerrit.wikimedia.org/r/256144

aaron closed this task as Resolved.Dec 21 2015, 9:47 PM

Assuming this is fixed in MW (unless something was missed).

Denniss removed a subscriber: Denniss.Apr 24 2016, 10:59 AM