Page MenuHomePhabricator

Update Dashiki Cloud Instances
Closed, ResolvedPublic

Description

We use our two Dashiki Cloud Instances for:

  • dashboards based on ReportUpdater (such as browser statistics)
  • staging environment for Wikistats (to test on real mobile devices before deploy)
  • general purpose web servers (hackathons, interview questions, etc)

We have marked these as "In Use" on the Cloud team's cleanup page: https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2024_Purge. The next steps are for us to do add two new instances based on the latest supported Debian image, and migrate. I (milimetric) unfortunately can not find my notes from the last migration. Some useful information is on Wikitech.

Event Timeline

VirginiaPoundstone subscribed.

Deadline is end of June 1st. Deletion on July 1st.

Keeping track of how I do this for future reference. (The previous task where I did this was T236586 and I failed to take good notes there)

  • (see this guide for accessing your instances, basically ssh dashiki-0X.dashiki.eqiad1.wikimedia.cloud)
  • launch new instances (cores1.ram2.disk20 seemed fine so far but I did cores2.ram4.disk20 in case bookworm needs more "juice")
  • attach the dashiki-transfer volume to old instance you're transferring from (I mount it to /srv/transfer, see cinder volume docs)
  • back up
    • sudo cp -rp /etc/apache2/sites-available /srv/transfer/ (apache config)
    • sudo cp -rp /srv/static /srv/transfer/ (static sites)
  • Set up Apache on the new machines
    • sudo apt install apache2
    • install any missing modules, as of this writing, these two:
      • apache2ctl -M on both machines will tell you what modules are different
      • a2enmod headers
      • a2enmod rewrite
  • restore
    • detach dashiki-transfer from old machine, attach to new machine, mount (usually sudo mkdir /srv/transfer && sudo mount /dev/sdb /srv/transfer)
    • restore files backed up above (you may have to edit the apache configs you move, for example I had to change ServerNames from wmflabs to wmcloud)
    • add symlinks for apache sites restored to sites-available (eg. sudo ln -s /etc/apache2/sites-available/50-simplestatic.conf /etc/apache2/sites-enabled/)
    • remove default symlinks, like 000-default...
    • restart apache server sudo systemctl restart apache2
  • move proxies https://horizon.wikimedia.org/project/proxy/
    • copy name, delete old, create new
    • (you might have to find the instance you're moving from by ip)

The two instances have been moved, docs updated on wiki and in code, and proxies have been moved. The only problem is the new proxies can't use the old wmflabs.org domain. For now, I left the old proxies up and additionally set up the new proxies. So, for example, both https://pingback.wmflabs.org/ and https://pingback.wmcloud.org/ work. Whenever the old instances are deleted, the old URLs will stop working. I guess part of sign-off will be to communicate this and maybe delete the old instances?