We would like to host Zim files on WMF infrastructure for the upcoming Android compilations feature.
On average, these files will vary in size from 1 GB - 20 GB.
Clients will use HTTP to download and will need support for byte offsets and ranges… and/or chunking.
- Where can we host these files?
- What caching considerations do we need to make?
- What about CDNs?
- Do we host any existing large files that we can use as a template?
So far we have 2 candidates for where to host:
- Swift (along with Commons media)
For example, Kiwix hosts its Zim files on their own server, with various academic and private mirrors in Europe and other continents; MirrorBrain directs every download to a suitable mirror (the dumps host serves only a negligible part of the total bandwidth):