codfw1dev: glance_rsync_images is failing
Closed, InvalidPublic
Actions

Assigned To

None

Authored By

	aborrero
	Oct 3 2019, 10:33 AM

Description

Something is not working in the glance rsync setup.

root@cloudcontrol2001-dev:~# systemctl status glance_rsync_images
● glance_rsync_images.service - Copy Glance images to standby server
   Loaded: loaded (/lib/systemd/system/glance_rsync_images.service; static; vendor preset: enabled)
   Active: active (running) since Thu 2019-10-03 10:28:22 UTC; 6s ago
 Main PID: 63256 (rsync)
    Tasks: 2 (limit: 39321)
   CGroup: /system.slice/glance_rsync_images.service
           ├─63256 /usr/bin/rsync -aSO /srv/glance/images/ cloudcontrol2003-dev.wikimedia.org:/srv/glance/images/
           └─63257 ssh cloudcontrol2003-dev.wikimedia.org rsync --server -lOogDtprSe.iLsfxC . /srv/glance/images/

Oct 03 10:28:22 cloudcontrol2001-dev systemd[1]: Started Copy Glance images to standby server.
Oct 03 10:28:22 cloudcontrol2001-dev rsync[63256]: rsync: failed to set permissions on "/srv/glance/images/.": Operation not permitted (1)

Not sure if related to T228972: CloudVPS: codfw1dev: refresh glance images

Details

	Subject	Repo	Branch	Lines +/-
	openstack: set glance image dir permissions on controllers	operations/puppet	production	+10 -19
	openstack: set glance image dir permissions on controllers	operations/puppet	production	+6 -8

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	aborrero	T217891 CloudVPS: rework codfw deployments
Resolved	None	T229441 CloudVPS: codfw1dev: missing bits
Invalid	None	T234518 codfw1dev: glance_rsync_images is failing

Event Timeline

aborrero created this task.Oct 3 2019, 10:33 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 3 2019, 10:33 AM

aborrero triaged this task as Low priority.Oct 3 2019, 10:34 AM

aborrero moved this task from Inbox to Soon! on the cloud-services-team (Kanban) board.

aborrero added a parent task: T229441: CloudVPS: codfw1dev: missing bits.

Change 540607 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540607

gerritbot added a project: Patch-For-Review.Oct 3 2019, 1:51 PM

Puppet is not managing the permissions on the secondary controllers image directory.

cloudcontrol2003-dev:~# ls -lad /srv/glance/images
drwxr-xr-x 2 glance glance 4096 Oct  1 21:29 /srv/glance/images

I believe the permissions to that directory were previously updated manually and remained in place since puppet wasn't managing it.

Change 540607 merged by Jhedden:
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540607

Maintenance_bot removed a project: Patch-For-Review.Oct 3 2019, 2:10 PM

Change 540612 had a related patch set uploaded (by Jhedden; owner: Jhedden):
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540612

gerritbot added a project: Patch-For-Review.Oct 3 2019, 2:48 PM

Change 540612 merged by Jhedden:
[operations/puppet@production] openstack: set glance image dir permissions on controllers

https://gerrit.wikimedia.org/r/540612

Maintenance_bot removed a project: Patch-For-Review.Oct 3 2019, 4:10 PM

this will be moot once we're using ceph for images, so might be worth ignoring in the meantime

this sync isn't run or needed anymore.

• Bstorm awarded a token.Oct 22 2020, 3:08 PM