Page MenuHomePhabricator

There should be a hook to get Squid/Varnish purge request URL for extension development
Open, LowPublic

Description

We know that Wikipedia generally don't use any third party CDN service. However, it is quite common for other wiki site (e.g. we the Moegirlpedia) to use commercial CDN service (Akamai/Cloudflare etc...). By far, there is no third party cache/proxy purge support available in Mediaiwki. Not even a hook allow extensions to get the page purge request.

https://www.mediawiki.org/wiki/Manual:CloudFlare
wrote that

HTTP purge
When new content is uploaded to a MediaWiki, the wiki software will request that each cache server (as configured in $wgSquidServers) discard the outdated version of the page or image. This notification is not sent to third-party web proxies (such as those listed in Extension:TrustedXFF) and appears to be hard-coded in SquidPurgeClient.php in core MediaWiki code with no hooks to allow an extension to change this behavior.

We wish Mediawiki to add a new hook, which allows extension developer to catch the purge request url, then send the request to commercial CDN service via their API system.

There was a similar hook available, however it was removed completely in Mediawiki version 1.17.0.
https://www.mediawiki.org/wiki/Manual:Hooks/HTMLCacheUpdate::doUpdate

To be more specific,
We know that $wgSquidServers setting is able to send standard Squid/Varnish purge request format PURGE http://wiki.example.org/Page. However, most commercial CDN service have their own purge request API system, and refuse to honor the standard format purge request.
Therefore, to create a extension, which implement third party commercial CDN purge function. The extension need to acquire all the URL of pages need purge, then convert the request to specific API format, then send to CDN servers.

We are developing Mediawiki extension that send purge request to Cloudflare. But there is no internal ways to get url of purge request from Mediawiki.

Therefore, a Mediawiki hook, which can pass all the purge request url to extensions, is needed.

Event Timeline

Zoglun created this task.Feb 15 2019, 5:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 15 2019, 5:52 AM
Zoglun updated the task description. (Show Details)Feb 15 2019, 6:09 AM

I believe you can use the EventRelayer interface for this. It does not yet have much documentation, but the following should get you in the right direction.

For a custom CDN purger:

  • Enable $wgUseCDN so that CdnCacheUpdate runs. (Keep these off $wgCdnReboundPurgeDelay, $wgCdnServers, and $wgHTCPRouting).
  • Configure the cdn-url-purges channel $wgEventRelayerConfig with an implementation that purges your CDN in the way that it requires.

See also:

The cdn-url-purges channel will be notified with a list of urls to purge in the CDN. As a site administrator, you'd configure $wgEventRelayerConfig['cdn-url-purges']. The extension could provide an EventRelayer subclass that handles messages for $channel === 'cdn-url-purges' in the way that your CDN provider requires. For example:

$wgUseCDN = true;
$wgEventRelayerConfig['cdn-url-purges'] = [
  'class' => MediaWiki\Extension\MyCloudFare\EventRelayer::class,
];
Krinkle triaged this task as Low priority.Jul 15 2019, 11:23 PM
Krinkle moved this task from Backlog to HTTP Cache proxy on the MediaWiki-Cache board.
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.