Page MenuHomePhabricator

Generate IPFS snapshots (via HTML dumps)
Open, Needs TriagePublic

Description

Wikipedia-on-IPFS [WPFS] is useful in a number of contexts. It offers a very fast way to share access to an offline snapshot, and censorship resistance. An instance of this was set up last year: https://blog.ipfs.io/24-uncensorable-wikipedia/
Making regular dumps available in this format would allow regular updates of WPFS, and significantly increase the number of full mirrors of Wikipedia.

This requires an HTML version of a dump. The IPFS update could be done directly from an uncompressed HTML dump on WM servers.

Event Timeline

Sj created this task.Nov 26 2018, 4:04 PM
Sj updated the task description. (Show Details)
Sj added a comment.Nov 26 2018, 6:44 PM

See also
https://phabricator.wikimedia.org/T182351 (the standing request for HTML dumps)
https://phabricator.wikimedia.org/T133547 (for a requested generation process)

Sj renamed this task from Generate HTML dumps again, for IPFS-friendliness to Generate IPFS snapshots (via HTML dumps) .Nov 26 2018, 6:46 PM
Sj updated the task description. (Show Details)

The test instance currently causes trouble: https://github.com/ipfs/distributed-wikipedia-mirror/issues/55 https://github.com/ipfs/distributed-wikipedia-mirror/issues/48 .

Is there a way to prevent the proposed snapshot from being put on the Internet/WWW? As far as I can tell, such a mirror makes sense only if it's on an intranet or its own non-HTTP protocol, otherwise it just adds to live mirrors which are legion already. The blog post said "Coming soon: We’ve almost finished creating a web browser extension that will allow you to access IPFS directly from your web browser" but https://github.com/ipfs-shipyard/ipfs-companion says "Requests made via experimental protocols are re-routed to HTTP gateway (public or custom)" so I guess it's not ready.