Page MenuHomePhabricator

Ubuntu to Debian migration for maps-warper2 instance of the wikimaps warper
Closed, ResolvedPublicDec 18 2018

Description

For the wikimaps warper: http://warper.wmflabs.org/

Because of https://phabricator.wikimedia.org/T204506

We need to:

✔️ Get a new instance running Debian for maps-warper2 (maps-warper3 ?)
✔️ Install the necessary software
✔️ Alter and configure mapwarper and code, libraries etc
✔️ Export and import database
✔️  Tie up everything with data   / backups
 ✔️  Cross fingers it all works!

Alternatives seen from other projects include putting the application in a docker container.

This task is to track the progress of this.

Details

Due Date
Dec 18 2018, 11:00 PM

Event Timeline

Chippyy created this task.Dec 4 2018, 9:51 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 4 2018, 9:51 PM
Aklapper set Due Date to Dec 18 2018, 11:00 PM.Dec 5 2018, 10:23 AM
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptDec 5 2018, 10:23 AM

If you build the new instance in the eqiad1-r region then there will be no migration downtime for the instance since it will already be in the new location. If Horizon is not showing eqiad1-r as an allowed region for the maps project, @Andrew can fix that for you.

Yurik added a subscriber: Yurik.Dec 6 2018, 7:07 PM

After some extensive work with Docker containers, it seems they are much cleaner, more portable, and much easier to deploy approach, without any performance or other issues. I would highly recommend repackaging under Docker.

Old instance is now in new location (for neutron) and site is working as before.

Update 😃

Instance provisioned maps-warper3

Packages installed
(compiled gdal utility installed to /usr/local/bin/gdal/utilities as the apt package has a bug )

warper application installed

database dumped and restored to new server

All appears to be working fine. NOTE I switched over the proxy https://warper.wmflabs.org to maps-warper3 so the main application will be on the new server.

The old application running on https://warper-old.wmflabs.org just in case things are borked and we can go back, before switching it off. Worth giving it a few days for any kinks to show themselves.

TODO - check any crons, logrotates, custom configs and any other stuff I can think of which might need migrating or different from ubuntu to debian.

for the record. https://phabricator.wikimedia.org/T208406 is relevant to maps-warper3 instance as we store around 230G worth of images in /home/warperdata

We could certainly have a local disk, but it would need to be big enough to accommodate new maps being added. I think 500G should be enough. At the very least, the NFS could use the project store without /home being mapped to the NFS which is a bit silly. Anyhow, priority here is to migrate from ubuntu instance to debian instance.

Chippyy updated the task description. (Show Details)Dec 17 2018, 6:27 PM
Chippyy updated the task description. (Show Details)
Chippyy updated the task description. (Show Details)Dec 17 2018, 6:48 PM

I think all the logrotate and the crontabs are copied across also. Wouldn't mind a day before we remove the warper-old proxy and then we can turn off the old instance.

Chippyy updated the task description. (Show Details)Dec 19 2018, 7:34 PM

I've removed the warper-old proxy. and shut off the maps-warper2 instance. I'd like to keep it around for a week or so before deletion though.

Chippyy closed this task as Resolved.Jan 8 2019, 5:01 PM

Resolved. Have just deleted the old maps-warper2 instance as everything's been running okay for a while.