Page MenuHomePhabricator

Don't dump closed wikis
Closed, ResolvedPublic

Description

A number of projects were closed/deleted but are still dumped regularly, e.g. https://rm.wiktionary.org (https://dumps.wikimedia.org/rmwiktionary)

Those projects should probably not be dumped anymore.

The difficulty is to have an up to date list of such projects. A good start would be this category: https://meta.wikimedia.org/wiki/Category:Approved_proposals_for_closing_wikis or this page: https://meta.wikimedia.org/wiki/Proposals_for_closing_projects.

Event Timeline

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 19 2016, 9:48 PM
Reedy assigned this task to ArielGlenn.Apr 24 2016, 11:07 AM
Reedy removed ArielGlenn as the assignee of this task.
Reedy triaged this task as Low priority.
Reedy added a subscriber: ArielGlenn.

Why not dump them? They aren't so big that the add any appreciable delay to the runs, afaict. And this way anyone who wants a copy of a closed wiki has a dump that's up to date with the latest MediaWiki version for import.

Most closed wikis are basically empty, I don't see any use for their dump for anyone, up to date or not.

If they are not costly to dump, then that's OK.

I just wish there was an easy way to to flag those mostly useless dumps as *closed wiki*, rather than looking at the hard to reach page that @Peachey88 mentioned.

Where would you want such an indicator to show up? That might be doable with not too much effort.

Some wikis have a "(private data)" in https://dumps.wikimedia.org/backup-index.html: a similar "(closed wiki)" label would work.

Change 285187 had a related patch set uploaded (by ArielGlenn):
mark closed wikis as such on the main index.html page for downloads

https://gerrit.wikimedia.org/r/285187

ArielGlenn moved this task from Backlog to Up Next on the Dumps-Generation board.

Change 285187 merged by ArielGlenn:
mark closed wikis as such on the main index.html page for downloads

https://gerrit.wikimedia.org/r/285187

ArielGlenn added a comment.EditedApr 26 2016, 1:08 PM

This will happen for all new dumps of closed wikis going forwards. I did a manual test on one wiki which can be seen marked as 'closed' on the main download page.

ArielGlenn moved this task from Up Next to Active on the Dumps-Generation board.Apr 26 2016, 2:23 PM
ArielGlenn closed this task as Resolved.May 5 2016, 3:46 PM

I see wikis showing up in the main index page marked as closed, as expected, so closing this task.

ArielGlenn moved this task from Active to Done on the Dumps-Generation board.May 5 2016, 3:46 PM