Page MenuHomePhabricator

Publish dumps on CDN
Open, LowPublicFeature

Description

Could the content of https://dumps.wikimedia.org/ be published on Content Delivery Networks?

Advantages:

  • Speed: Users (especially those outside the United States) could download dumps faster than today because dumps would be served from the CDN edge.
  • Saves bandwidth for Wikimedia infrastructure.
  • Trivial to implement: uploading a file to a CDN is a single command-line invocation. There’s a wide selection of open-source tools to upload files to CDNs that compatible with the Amazon S​3 API.

Disadvantages:

  • Dependency on commercial supplier. Proposed mitigation: Use multiple suppliers.
  • Cost of the CDN. Proposed mitigation: Talk to the supplier’s marketing departments. They’ll probably be glad to partner with Wikimedia as a showcase for their products, and offer the hosting for free / as a donation.
  • Politics. If WMF were to partner with the likes of Cloudflare, Amazon, Google or Microsoft, this might raise eyebrows. Proposed mitigation: (a) use multiple CDNs, not just a single one, including smaller companies in this space not just the industry giants; (b) careful messaging and explanation; (c) allow any CDN to partner with WMF at no cost for either side, and explain this in a public blog post.