Page MenuHomePhabricator

Ubuntu to Debian migration for maps-warper2 instance of the wikimaps warper
Closed, ResolvedPublicDec 18 2018

Description

For the wikimaps warper: http://warper.wmflabs.org/

Because of https://phabricator.wikimedia.org/T204506

We need to:

✔️ Get a new instance running Debian for maps-warper2 (maps-warper3 ?)
✔️ Install the necessary software
✔️ Alter and configure mapwarper and code, libraries etc
✔️ Export and import database
✔️  Tie up everything with data   / backups
 ✔️  Cross fingers it all works!

Alternatives seen from other projects include putting the application in a docker container.

This task is to track the progress of this.

Details

Due Date
Dec 18 2018, 11:00 PM

Event Timeline

Aklapper set Due Date to Dec 18 2018, 11:00 PM.Dec 5 2018, 10:23 AM
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptDec 5 2018, 10:23 AM

If you build the new instance in the eqiad1-r region then there will be no migration downtime for the instance since it will already be in the new location. If Horizon is not showing eqiad1-r as an allowed region for the maps project, @Andrew can fix that for you.

After some extensive work with Docker containers, it seems they are much cleaner, more portable, and much easier to deploy approach, without any performance or other issues. I would highly recommend repackaging under Docker.

Old instance is now in new location (for neutron) and site is working as before.

Update 😃

Instance provisioned maps-warper3

Packages installed
(compiled gdal utility installed to /usr/local/bin/gdal/utilities as the apt package has a bug )

warper application installed

database dumped and restored to new server

All appears to be working fine. NOTE I switched over the proxy https://warper.wmflabs.org to maps-warper3 so the main application will be on the new server.

The old application running on https://warper-old.wmflabs.org just in case things are borked and we can go back, before switching it off. Worth giving it a few days for any kinks to show themselves.

TODO - check any crons, logrotates, custom configs and any other stuff I can think of which might need migrating or different from ubuntu to debian.

for the record. https://phabricator.wikimedia.org/T208406 is relevant to maps-warper3 instance as we store around 230G worth of images in /home/warperdata

We could certainly have a local disk, but it would need to be big enough to accommodate new maps being added. I think 500G should be enough. At the very least, the NFS could use the project store without /home being mapped to the NFS which is a bit silly. Anyhow, priority here is to migrate from ubuntu instance to debian instance.

Chippyy updated the task description. (Show Details)

I think all the logrotate and the crontabs are copied across also. Wouldn't mind a day before we remove the warper-old proxy and then we can turn off the old instance.

I've removed the warper-old proxy. and shut off the maps-warper2 instance. I'd like to keep it around for a week or so before deletion though.

Resolved. Have just deleted the old maps-warper2 instance as everything's been running okay for a while.