Page MenuHomePhabricator

Remove puppet class role::labs::lvm::mnt
Closed, DeclinedPublic


WMCS instances have some extra disk space which is not mounted by default. It can be mounted on /srv using role::labs::lvm::srv. There is a legacy class role::labs::lvm::mnt (which mount it on /mnt) which is deprecated and should be removed entirely.

Continuous integration was among the early adopters of /mnt. They all got migrated to /srv via T146381.

Labs instances should be switched to role::labs::lvm::srv @hashar wrote a migration guide on :

  • switch from role::labs::lvm::mnt to role::labs::lvm::srv
  • run puppet
  • kill anything that might be using /mnt (lsof /mnt)
  • manually umount /mnt
  • ensure /srv is mounted
  • VERIFY /mnt is no more refered to in /etc/fstab
    • failure to do so would cause the instance to be deadlocked on the next reboot


As of October 20 2017:

Project: deployment-prep Beta-Cluster-Infrastructure - DONE Oct 30th

  • deployment-logstash2.deployment-prep.eqiad.wmflabs

Project: tools PAWS

  • tools-paws-*

Project: wikidata-ldf Wikidata

  • wikidata-ldf.wikidata-ldf.eqiad.wmflabs

Event Timeline

deployment-logstash2.deployment-prep.eqiad.wmflabs has three mounts:

/dev/vd/second-local-disk  /srv                    ext4  defaults  0  0
/dev/vd/second-local-disk  /var/lib/elasticsearch  ext4  defaults  0  0
/dev/vd/second-local-disk  /mnt                    ext4  defaults  0  0

Mentioned in SAL (#wikimedia-releng) [2017-10-30T10:56:20Z] <hashar> deployment-logstash2 removed puppet class role::labs::lvm::mnt, replacing with role::labs::lvm::srv . /srv is already mounted. Unmounting /mnt and restarting elastcisearch - T178722

Hello @CristianCantoro

Looks like you have created the instance wikidata-ldf.wikidata-ldf.eqiad.wmflabs. It is apparently mounting the instance extended disk space on /mnt and I would like to get it normalized to use /srv. That is due to the puppet class role::labs::lvm::mnt being applied to it. This task has some instructions to do the migration, but you would also need to adjust any code/settings that refers to /mnt.

Would it be possible to migrate it? I am available to assist.

hashar triaged this task as Medium priority.Nov 9 2017, 2:01 PM

PAWS uses role::labs::lvm::mnt with the mount_point being directed at /var/lib/docker instead of /mnt. Role role::labs::lvm::srv does not allow for that.
To me it seems the sanest path to remove role::labs::lvm::mnt would be develop a PAWS specific role that includes either a mountpoint definition in that path or, ideally, a PAWS specific role that installs docker, requires ::profile::labs::lvm::srv and adds -g /srv/docker to the docker default file.

It looks like only one instance still uses this: osmit-uno.osmit.eqiad.wmflabs in the osmit project, based on
Well, and PAWS, actually.

Indeed that is the same, still tools / paws. From:

Project: tools


Project: toolsbeta


The original intent was to phase out /mnt cause at the time we settled at using /srv instead. I have done it for a few projects I could act on and some other disappeared with time. The left over seems to work as is and it is overall not that important to switch to /srv. It can potentially be taken in mind if paws get refactored at some point, but meanwhile there is no need for immediate action. Hence closing the task to make my backlog slightly smaller ;)