Page MenuHomePhabricator
Create Task
Maniphest T16584

New API to give latest dump file release
Closed, InvalidPublic
Actions

Description

It is beyond what the API currently does but it's also on topic and might be worthwhile.

Currently to programatically check whether a new dump file is available you need to resort to screen scraping. To do so from JavaScript is even tricker because of the cross-domain policy.

The minimum requirement of the API would just return the ASCII date of the most recent fully successful dump for the wiki. For example the dump file I'm currently using for en.wiktionary is "20080613".

Version: unspecified
Severity: enhancement
URL: http://download.wikipedia.org/enwikipedia/latest/

Details

Reference
bz14584

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 21 2014, 10:13 PM
• bzimport added a project: Datasets-General-or-Unknown.
• bzimport set Reference to bz14584.
Hippietrail created this task.Jun 18 2008, 4:23 PM
Catrope added a comment.Jun 18 2008, 7:34 PM

I assume this is about http://download.wikipedia.org/backup-index.html ? In my opinion, download.wikipedia.org itself is responsible for offering a proper bot interface. There's no communication from enwiktionary to download.wikipedia.org, only the latter grabbing data from the former and tarballing it, so there's not really a way for enwiktionary to know when it was last successfully dumped (other than screenscraping behind the scenes, which would just move the problem).

Moving this bug to the Wikimedia product, as it's for Wikimedia's dump facility to address, not the MediaWiki software itself.

• brion added a comment.Jul 5 2008, 12:20 AM

Use the RSS feeds for this; they're in the latest dirs alongside the symlinks.

Hippietrail added a comment.Jul 5 2008, 1:53 PM

Adding URL to the "latest" dir for enwikipedia since they are not publicized or linked to from anywhere and thus very hard to find.

ArielGlenn moved this task from Backlog to Done on the Datasets-General-or-Unknown board.Jul 14 2016, 10:19 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL