Page MenuHomePhabricator

Prepare for puppetizing /etc/apt/sources.list
Open, LowPublic

Description

https://gerrit.wikimedia.org/r/c/operations/puppet/+/631396 puppetizes /etc/apt/sources.list. That change is currently not active on cloud-vps.

It's probably better to enable on cloud-vps as well, but first we need to locate VMs that have custom repos added by hand and move those configs into /etc/apt/sources.list.d/<whatever>

Event Timeline

Andrew created this task.Oct 1 2020, 1:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2020, 1:46 PM

For a Buster system, the puppetised sources.list looks like this:

deb http://mirrors.wikimedia.org/debian/ buster main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ buster main contrib non-free

deb http://security.debian.org/debian-security buster/updates main contrib non-free
deb-src http://security.debian.org/debian-security buster/updates main contrib non-free

# -updates, previously known as 'volatile'
deb http://mirrors.wikimedia.org/debian/ buster-updates main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ buster-updates main contrib non-free

For a Stretch system, the puppetised sources.list looks like this:

deb http://mirrors.wikimedia.org/debian/ stretch main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ stretch main contrib non-free

deb http://security.debian.org/debian-security stretch/updates main contrib non-free
deb-src http://security.debian.org/debian-security stretch/updates main contrib non-free

# -updates, previously known as 'volatile'
deb http://mirrors.wikimedia.org/debian/ stretch-updates main contrib non-free
deb-src http://mirrors.wikimedia.org/debian/ stretch-updates main contrib non-free

I wonder if there are CloudVPS VMs in which the admin hand-edited the file. The moment we enable this, the changes would be lost, right?

Andrew added a comment.Oct 1 2020, 6:32 PM

I wonder if there are CloudVPS VMs in which the admin hand-edited the file. The moment we enable this, the changes would be lost, right?

Yes. This task is about detecting and counting those VMs and moving the customisations out of the way of the puppet management.

I wonder if there are CloudVPS VMs in which the admin hand-edited the file. The moment we enable this, the changes would be lost, right?

Yes. This task is about detecting and counting those VMs and moving the customisations out of the way of the puppet management.

Exactly. I'd expect that 99% of all VMs don't have anything in there, but there are surely some edge cases. OTOH, overwriting /etc/apt/sources.list does not cause any imminent breakage, all packages remain installed as they are, it only affects further updates. As such, it's probably also fine to simply announce this to users, give them a week and then merge a patch?

Andrew added a comment.Oct 9 2020, 5:54 PM

A few things I'm seeing:

  • There's a huge amount of diversity from host to host in this file
  • Out of 700+ VMs, only ONE of them refers to mirrors.wikimedia.org in sources.list
  • Relatively many (dozens) of VMs have refs to download.docker.com in sources.list

This chaos makes me really want to normalize things, and also makes me quite nervous about normalizing.

nskaggs removed Andrew as the assignee of this task.Oct 20 2020, 4:30 PM
nskaggs triaged this task as Low priority.