Page MenuHomePhabricator

Rebuild Routinator (rpki) VMs with larger disk
Open, LowPublic

Description

As discussed in T291543, the current VMs for rpki1001 and rpki2001 are a little undersized, operating close to their capacity and sometimes exhausting inodes when the service restarts.

Current drive is 10GB, recommendation is to rebuild them with 20GB disk to avoid any future niggles.

Creating this task to track progress.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

https://packages.nlnetlabs.nl/ also provides the routinator debs for bullseye (plus it's a static Go binary anyway), so if we're recreating the VMs anyway, let's also switch to Bullseye?

@ayounsi Riccardo suggested maybe using a separate disk/partition for the routinator data? That was partly to just do a quick dirty job and not rebuild, but we've reason to rebuild anyway so let's do that.

Do you think it would still make sense to have a separate disk/partition for the Routinator data?

@ayounsi Riccardo suggested maybe using a separate disk/partition for the routinator data? That was partly to just do a quick dirty job and not rebuild, but we've reason to rebuild anyway so let's do that.

Do you think it would still make sense to have a separate disk/partition for the Routinator data?

The files in /var/lib/routinator is all fairly small (we have >400k files with 2.8G in total on rpki1001), in theory we could create create a custom partman config with fstype=small or news, but OTOH we easily have the extra disk space available and have spent quite some time to reduce our maze of Partman configs (https://phabricator.wikimedia.org/T156955)

So unless we expect RPKI usage (and the data used by routinator) to grow massively over time, so my suggestion would be to simply go with the extra disk usage.

Change 726610 had a related patch set uploaded (by Muehlenhoff; author: Muehlenhoff):

[operations/puppet@production] Add repo sync definition and repo component for Routinator

https://gerrit.wikimedia.org/r/726610

Change 726610 merged by Muehlenhoff:

[operations/puppet@production] Add repo sync definition and repo component for Routinator

https://gerrit.wikimedia.org/r/726610

Mentioned in SAL (#wikimedia-operations) [2021-10-05T15:10:22Z] <moritzm> imported routinator 0.10.1-1bullseye to thirdparty/routinator for bullseye-wikimedia T292503

I've added routinator to apt.wikimedia.org at "thirdparty/routinator" for bullseye-wikimedia and adapted the Puppet code, so that when the these get reinstalled with Bullseye, the thirdparty component is picked.