networkx has some breaking API changes between 1.x and 2.x which are non-trivial to resolve. To unbreak the use of cergen on buster the build has been adapted to use a forward-ported 1.11 package on a separate component for buster-wikimedia (component/cergen, which now also includes cergen itself).
Thanks, could we try upgrading the BIOS/firmware initially on 2002? Maybe tomorrow (I'd prepare the server so that it can be taken down without impact)?
Fri, Oct 11
Can we narrow down which component needs libssl1.0.0? One of the many outdated/bundled ones?
Thu, Oct 10
I looked into this and it's quite a mess!
The package was already previously copied between jessie and stretch, has no runtime dependencies which need to be updated and the if-up.d/if-down.d interfaces are unchanged between stretch/buster, so I simply copied over the deb from stretch-wikimedia to buster-wikimedia.
Wed, Oct 9
One other thing (not necessarily now) is to add a monitoring check, e.g. https://exchange.nagios.org/directory/Plugins/Security/check_krb5
Another thing we need to do: Add a new flag to data.yaml to annotate that a user is kerberos-enabled (as we need to ensure to also drop Kerberos user principals when offboarding users).
There are still Puppet references towards labpuppetmaster* in Puppet (e.g. hieradata/eqiad/profile/openstack/eqiad1/puppetmaster.yaml, please remove these fully before handing over for reclaim.
Tue, Oct 8
See earlier discussion on task, this is still used by Toolforge, so WMCS SREs might still want to tweak the log spam.
I see in dmesg that it got removed from auth1001, but I don't see it in the logs for auth1002, is the USB slot in question maybe inactive? Could you try moving it to a different slot?
Mon, Oct 7
bdsync was never packaged in Debian, it's an internally packaged tool (originally done by Chase), from a quick glance rebuilding it for buster should be straightforward.
- Grafana is installed from an external repository. There's already a config to pull in the new deb package for our Buster repository (Chris is working on setting up a Grafana 6 instance on Buster), but it hasn't been imported to our apt.wikimedia.org repository yet. Best to sync up with him on that, as I'm not 100% sure whether it's best to import grafana 5 or 6 initially.
Wed, Oct 2
JFTR; the task description is inaccurate, supports ends five years after the jessie release, i.e. April 25 (but the internal hard deadline is end of Q3 as we still need time to properly wrap things up in puppet/repos etc.)
This is fixed for Buster and Stretch, the remaining ~ 100 Jessie hosts won't get fixed, they'll vanish in the next six months anyway.
Tue, Oct 1
7.2.22 is rolled out fleet-wide to all servers using PHP 7.2
There's two issues with the patch merged for Erin Yener: (1) If contractors have a @wikimedia.org address, they should be added to cn=wmf, not cn=nda. (2) Contractors need an entry in data.yaml with the contract end and a person of contact (expiry_date, expiry_contact fields). Otherwise we'll miss dropping their credentials when the contract expires (we ping the point of contact one week before the contract expires and will extend access if the contract is contuining)
Mon, Sep 30
But nodejs on the proton* hosts is still on nodejs 6?
@Dzahn: The current update for PHP 7.2.22 is a little special as there was an upstream change in the shipped default conffile (a new option for sqlite was added). You can use the following via Cumin:
Fri, Sep 27
I've built the 2.4.2pre package for stretch-wikimedia and tested it on a few servers successfully (also comparing iptables output to spot any potential regression from the 2.3->2.4 move). I'll upload that on Monday as it needs some coordination with Arturo to syncronise the rollout to Cloud VPS (as the ferm package ships ferm.conf as a conffile, to rule out issues with unattended-upgrades overwriting the puppetised version)
John and I have discussed next steps on IRC: Initially we'll make U2F opt-in via a memberOf/LDAP check. At a later step we'll add TOTP support (ideally in a way that allows to import the existing registrations from the wikitech endpoint) and by then we'll need MFA selection either by means of the Groovy script or via the selector support included in 6.1: https://apereo.github.io/2019/05/13/cas61x-mfa-selection-strategies/
Ack, what I meant was using the Tomcat packages as shipped in Debian
These are now ready to be wiped/reclaimed as spares.
This was done a while ago, we've settled on Apereo CAS.
Thu, Sep 26
Maybe the NIC on the server broke? Are there some self-tests/diagnostics for that on the hardware side?