Page MenuHomePhabricator

python3-ldap3 mixed versions and future traps
Open, MediumPublic

Description

While trying to deploy https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/482237/ I realized that the python3-ldap3 library that we get from apt is really old. Newer versions of the library provide nice features like easy support for paged result sets. See this diff in gerrit for an example of the difference in lines of code needed for this common use-case.

Also we have different versions on various hosts due to some usage of backports and pinning. Both role::labs::db::maintain_dbusers and toollabs::maintain_kubeusers pin to the jessie backports package (0.9.9.3-1~bpo8+2) which is just slightly newer than the default jessie package (0.9.4.2-1).

Even more fun, newer versions of the library are not compatible with existing usage in several scripts. Once we start building things out on Buster (which has python3-ldap3 2.4.1-1) this is going to cause problems with the following scripts:

  • modules/grafana/files/grafana_auth_ldap_migrate.py
  • modules/ldap/files/reset-ldap-password
  • modules/profile/files/wmcs/nfs/maintain-dbusers.py
  • modules/toollabs/files/maintain-kubeusers
  • modules/toollabs/files/toolviews.py

openstack::puppet::master::enc also installs python3-ldap3, but it does not seem to be used directly there. This appears to be leftover from an earlier version of the modules/profile/files/puppetmaster/labs-puppet-enc.py script (rOPUPc658ac3d3795).

Event Timeline

Change 547700 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: Update toolviews.py for ldap3 v2.4.1

https://gerrit.wikimedia.org/r/547700

Change 547700 merged by Phamhi:
[operations/puppet@production] toolforge: Update toolviews.py for ldap3 v2.4.1

https://gerrit.wikimedia.org/r/547700

Bstorm triaged this task as Medium priority.Feb 11 2020, 5:43 PM

There's nothing incompatible that I can find in our usage. modules/profile/files/wmcs/nfs/maintain-dbusers.py uses methods that still work on later versions. They are just really ugly. On the other hand, old versions of ldap3 in python do *not* work with python 3.7+. I ran into this today with https://github.com/toolforge/toolsctl

That said, we don't have many stretch things running this stuff now other than nfs servers.

Out of that list I think maintain-dbusers is the only remaining issue.