Page MenuHomePhabricator

puppet vs. stretch vs. keystone
Closed, ResolvedPublic

Description

On cloudcontrol2001-dev, keystone is sometimes running and sometimes not... it seems like a coin-toss. Explicitly starting it with 'service keystone start' generally does not result in it running.

It's also the case that puppet never settles down into a stable state on that system. I suspect these facts are related.

P8542

I've confirmed that puppet runs are stable on cloudcontrol1003 which has a similar catalog but for Jessie.

Event Timeline

Andrew created this task.May 19 2019, 3:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 19 2019, 3:23 PM
Andrew updated the task description. (Show Details)May 19 2019, 3:26 PM

This seems to be some kind of conflict in the packages? between python-ldap and keystone. I will investigate.

aborrero triaged this task as Normal priority.May 20 2019, 12:07 PM
aborrero moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.

I originally tried to solve this with this patch https://gerrit.wikimedia.org/r/c/operations/puppet/+/500014 because python-pyldap was a fork of python-ldap.

It turns out that python-ldap is installed also by:

class ldap::client::utils($ldapconfig) {
    require_package('python-ldap', 'python-pycurl')
aborrero@cumin1001:~ $ sudo cumin "P{R:Class = ldap::client::utils}"
15 hosts will be targeted:
cloudcontrol[2001,2003]-dev.wikimedia.org,cloudcontrol[1003-1004].wikimedia.org,cloudservices[1003-1004].wikimedia.org,cloudstore[1008-1009].wikimedia.org,labstore[1003-1005].eqiad.wmnet,labweb[1001-1002].wikimedia.org,mwmaint2001.codfw.wmnet,mwmaint1002.eqiad.wmnet
DRY-RUN mode enabled, aborting

Since python-ldap can be replaced by python-pyldap my proposal is to simply drop python-ldap from ldap::client::utils.

But wait! this seems to be a bit more complex:

14:30 <arturo> hey zigo, can you share some hints on `python-pyldap` vs `python-ldap`?
14:30 <zigo> arturo: I can't remember the exact details, but basically, the fork got remerged.
14:30 <zigo> I can't remember what the direction is, sorry ...
14:30 <arturo> -_-
14:30 <zigo> But the one with version 3 has the re-merge.

So it seems we have another super special snowflake here.

I should investigate now how to deal with this combo... CC'ing @jbond and @MoritzMuehlenhoff just in case.

Change 511413 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] ldap::client::utils: don't require python-ldap if already declared

https://gerrit.wikimedia.org/r/511413

Change 511413 merged by Andrew Bogott:
[operations/puppet@production] ldap::client::utils: don't require python-ldap if already declared

https://gerrit.wikimedia.org/r/511413

Andrew closed this task as Resolved.May 20 2019, 3:59 PM

This seems better now!

Change 511460 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] keystone: set logdir group to 'keystone'

https://gerrit.wikimedia.org/r/511460

Change 511460 merged by Andrew Bogott:
[operations/puppet@production] keystone: set logdir group to 'keystone'

https://gerrit.wikimedia.org/r/511460