Page MenuHomePhabricator
Paste P6813

drafts of announcements for migration from dataset1001->labstore1006,7
ActivePublic

Authored by ArielGlenn on Mar 7 2018, 2:44 PM.
Subject: WMF dumps mirror site changing
To our mirror sites:
Thank you for continuing to mirror Wikipedia dumps and/or various other datasets over the years. We are about to migrate to a new server setup.
Server names and IP addresses: dataset1001.wikimedia.org and its associated IP addresses will be going away. Please use dumps.wikimedia.org after the migration date, which will automatically resolve to the correct host. If you would like to check network connectivity and routing with us before that date, please drop us an email, we're happy to oblige.
Dates: The new service will be available starting April 4 2018; the old service will be turned off one week later. The migration is planned to start at 14:30 UTC, and there may be a short downtime during the migration window.
Other changes: You should not need to do anything other than redirect your rsyncers to dumps.wikimedia.org.
Thank you again! We'll send more updates closer to the migration date. If you have any questions or comments, just reply to this email.
-------------------------------------
Subject: WMF datasets server migration
Hey Erik!
We're finally getting off the slow memory-and-network-limited dataset1001 host and moving to a couple of big beefy labstore boxes to host dumps and all other datasets. In the course of the migration, we're reorganizing all of our various rsyncs to/from the servers. So as part of that, we'd like to handle your rsync jobs on the labstore end, via cron on the server. We can run it once an hour if that works for you, and the idea is that you'd have a directory where you copy things in with the appropriate path and we pick them up with a straight rsync (no deletions) into the public/other/pageviews-ez directory tree.
We're doing this with all of our incoming datasets, for better management of all of these jobs.
For a short while your datasets would be going both to dataset1001 using your scripts, and to the new labstore box using our server-side rsyncs, until the dataset1001 service is finally turned off.
Dates: We would like to enable the server-side rsync service on March 28, 2018, and we can expect to turn off the crons pushing to dataset1001 by April 6th.
Please let us know soon what directory to pull from, still on stat1005, and if you have any comments or suggestions.
Thanks!
Your friendly WMF dump maintainers
----------------------------
Subject: WMF nfs service migration
Hey stats/analytics users!
The host currently providing all of your dumps and datasets needs over NFS is being replaced soon. All datasets will be available on a new server, which will be accessible at the same path as the old one, but there will be a transition time of a few hours. During that time, you may encounter stale data or the files may simply be inaccessible. Please schedule your work accordingly.
Dates: The migration is scheduled for April 2nd starting at 14:30 UTC, and is expected to last a few hours.
Thanks! We'll send more updates closer to the migration date. If you have any questions, just let us know.
-------
Subject: [Announce] /public/dumps NFS share maintenance on 2018-04-02
Dear Wikimedia Cloud VPS/Toolforge users,
If you use dumps/pageviews/other datasets available via /public/dumps on Toolforge/VPS instance, this email is for you!
The underlying NFS storage server is being replaced. Your access path will remain the same, but the data may be stale or inaccessible during the migration to the new servers.
Dates:
Dates: The migration is scheduled for April 2nd starting at 14:30 UTC, and is expected to last a few hours.
Thanks! We'll send more updates closer to the migration date. If you have any questions, just let us know.

Event Timeline

madhuvishy edited the content of this paste. (Show Details)Mar 7 2018, 9:50 PM
madhuvishy edited the content of this paste. (Show Details)Mar 7 2018, 11:27 PM
ArielGlenn edited the content of this paste. (Show Details)Mar 8 2018, 9:53 AM

For the web service migration, broader email blast:

Hello dumps.wikimedia.org users!

The servers that host the dumps.wikimedia.org site are being replaced with shiny new machines! The web service migration is set to happen at 14:30 UTC on April 4 2018. If you are trying to connect to dumps.wikimedia.org around the migration window, you might experience a short downtime. The switchover should ideally only take a few minutes, and we'll keep you posted once it's all done, or if anything changes!

As always, please feel free to reach out to us with any questions or concerns.

Best,

Madhumitha Viswanathan & Ariel Glenn
Wikimedia Foundation