Page MenuHomePhabricator

Keep wmflabs scoring boxes up-to-date
Closed, ResolvedPublic


We're running very old versions of Jessie,, apparently without regular security updates. I'm not sure exactly what role our wmflabs boxes play, but there's no reason to tempt fate by skipping updates.

Event Timeline

ArielGlenn triaged this task as Medium priority.Jun 26 2017, 9:15 AM
ArielGlenn added a project: Cloud-VPS.

Weird. We have been getting restarts when there's a labs-wide kernel update. What makes you think we aren't getting updates and what should we do to get them?

@Halfak they update the labs machine that hosts the vm. So the machines got updated but not the vms.

I'm surprised to find out that we aren't getting regular updates on these vms. Why has that happened and how do we change it?

The only way to change that is to run a cron script that does apt-get update and then apt-get upgrade -y

Ladsgroup changed the task status from Open to Stalled.Aug 2 2018, 5:43 PM
Ladsgroup subscribed.

It's blocked on T169247

Change 454546 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] wikilabels: Add zlib1g-dev package and cronjob to remove expired tasks

Mentioned in SAL (#wikimedia-cloud) [2018-08-22T18:12:55Z] <Amir1> redirecting traffic of from wikilabels-01.eqiad.wmflabs to wikilabels-02.eqiad.wmflabs (T168478)

ores nodes are read-only while wikilabels nodes are read/write and contain sensitive information (database credentials, OAuth credentials). I made a new node and migrated the credential files and puppetized everything else so we can throw away VMs faster in future. I made a temporarily DNS proxy to expose the new node (wikilabels-02) to the outside and ran tests on it, when I was sure it writes to the database and OAuth works fine, I redirected the traffic to the new node but I keep the old node alive for a week in case anything happens.

Change 454546 merged by Alexandros Kosiaris:
[operations/puppet@production] wikilabels: Add zlib1g-dev package and cronjob to remove expired tasks

All of the VMs now run Debian Stretch. Yay!