Page MenuHomePhabricator

Announce analytics.wikimedia.org/datasets and deprecation of datasets.wikimedia.org
Closed, ResolvedPublic5 Estimated Story Points

Description

UPDATE: need to update the documentation, for example https://wikitech.wikimedia.org/wiki/Analytics/Systems/Dashiki#How_to_get_your_data_available_via_http

Some content from datasets.wikimedia.org has been migrated to analytics.wikimedia.org/datasets. Namely all the automated reports that we know are produced with reportupdater. This is a long running task to migrate the rest of datasets.wikimedia.org and explain to people how files sync between our statistics servers (like stat1002 and stat1003). So two parts:

  • continue to clean these datasets until we think we can turn off datasets.wikimedia.org
  • announce that we're turning off datasets.wikimedia.org with plenty of warning, and then turn it off

Event Timeline

Nuria changed the point value for this task from 0 to 5.Mar 20 2017, 3:35 PM
Nuria moved this task from Parent Tasks to Next Up on the Analytics-Kanban board.

Latest thinking on this:

  1. Add a redirect from https://datasets.wikimedia.org to https://analytics.wikimedia.org/datasets/archive.
  2. Move any and all of aggregate-datasets, public-datasets, limn-public-data from stat1002 and stat1003 public folders into published-datasets/archive

These two in combination will mean all the old links will work and people can move/change the structure going forward as they see fit. And we all have a clean root folder with a nice README explaining that we should all try and keep things neat going forward.

If people do want to move/change, we should explain in the README and in the announcement that they should leave symlinks or redirects behind so as to not break any links.

I will bring this proposal to others using these folders (already checked with Aaron and Dario) and will look to make it happen within the week.

Change 360900 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove datasets.wikimedia.org site, redirect to analytics.wikimedia.org/datasets/archive

https://gerrit.wikimedia.org/r/360900

Change 360900 merged by Ottomata:
[operations/puppet@production] Remove datasets.wikimedia.org site, redirect to analytics.wikimedia.org/datasets/archive

https://gerrit.wikimedia.org/r/360900