Page MenuHomePhabricator

codfw1dev: glance_rsync_images is failing
Open, LowPublic

Description

Something is not working in the glance rsync setup.

root@cloudcontrol2001-dev:~# systemctl status glance_rsync_images
● glance_rsync_images.service - Copy Glance images to standby server
   Loaded: loaded (/lib/systemd/system/glance_rsync_images.service; static; vendor preset: enabled)
   Active: active (running) since Thu 2019-10-03 10:28:22 UTC; 6s ago
 Main PID: 63256 (rsync)
    Tasks: 2 (limit: 39321)
   CGroup: /system.slice/glance_rsync_images.service
           ├─63256 /usr/bin/rsync -aSO /srv/glance/images/ cloudcontrol2003-dev.wikimedia.org:/srv/glance/images/
           └─63257 ssh cloudcontrol2003-dev.wikimedia.org rsync --server -lOogDtprSe.iLsfxC . /srv/glance/images/

Oct 03 10:28:22 cloudcontrol2001-dev systemd[1]: Started Copy Glance images to standby server.
Oct 03 10:28:22 cloudcontrol2001-dev rsync[63256]: rsync: failed to set permissions on "/srv/glance/images/.": Operation not permitted (1)

Not sure if related to T228972: CloudVPS: codfw1dev: refresh glance images

Event Timeline

aborrero created this task.Thu, Oct 3, 10:33 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptThu, Oct 3, 10:33 AM
aborrero triaged this task as Low priority.Thu, Oct 3, 10:34 AM
aborrero moved this task from Inbox to Important on the cloud-services-team (Kanban) board.

Change 540607 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540607

Puppet is not managing the permissions on the secondary controllers image directory.

cloudcontrol2003-dev:~# ls -lad /srv/glance/images
drwxr-xr-x 2 glance glance 4096 Oct  1 21:29 /srv/glance/images

I believe the permissions to that directory were previously updated manually and remained in place since puppet wasn't managing it.

Change 540607 merged by Jhedden:
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540607

Change 540612 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540612

Change 540612 merged by Jhedden:
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540612