Page MenuHomePhabricator

Fix partitions on CI slaves , some are missing /var/lib/docker
Open, Needs TriagePublic

Description

Some integration-slave-docker instances have been provisioned with a /var/lib/docker partition but others do not. It is a bit confusing.

$ sudo cumin --trace --force 'name:docker' 'mount -t ext4|sort'
13 hosts will be targeted:
integration-slave-docker-[1021,1034,1037,1040-1041,1043,1048-1054].integration.eqiad.wmflabs
===== NODE GROUP =====                                                                                                                                                                                    
(3) integration-slave-docker-[1021,1049,1053].integration.eqiad.wmflabs                                                                                                                                   
----- OUTPUT of 'mount -t ext4|sort' -----                                                                                                                                                                
/dev/mapper/vd-second--local--disk on /srv type ext4 (rw,relatime,data=ordered)                                                                                                                           
/dev/vda3 on / type ext4 (rw,relatime,data=ordered)
===== NODE GROUP =====                                                                                                                                                                                    
(10) integration-slave-docker-[1034,1037,1040-1041,1043,1048,1050-1052,1054].integration.eqiad.wmflabs                                                                                                    
----- OUTPUT of 'mount -t ext4|sort' -----                                                                                                                                                                
/dev/mapper/vd-docker on /var/lib/docker type ext4 (rw,relatime,data=ordered)                                                                                                                             
/dev/mapper/vd-second--local--disk on /srv type ext4 (rw,relatime,data=ordered)                                                                                                                           
/dev/vda3 on / type ext4 (rw,relatime,data=ordered)
================

Or in short, the following instances lack a /var/lib/docker dedicated partition:

integration-slave-docker-1021.integration.eqiad.wmflabs
integration-slave-docker-1049.integration.eqiad.wmflabs
integration-slave-docker-1053.integration.eqiad.wmflabs

Maybe because they predate the introduction of /var/lib/docker and puppet is unable to magically shuffle the partitions for us. In which case we would have to provision new instances and delete those.

Event Timeline

hashar created this task.Apr 15 2019, 8:15 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 15 2019, 8:15 PM
hashar added a subscriber: dduvall.

That is set by profile::ci::dockervolume which comes from df1dc5957edce36b12909b3e8bd8cc1b8935fb44 by @dduvall

I have added two Jenkins slaves with role::ci::slave::labs::docker with $docker_lvm_volume = true. One had a sub partition /var/lib/docker the other did not :]