The Cloud-Services project tag is not intended to have any tasks. Please check the list on https://phabricator.wikimedia.org/project/profile/832/ and replace it with a more specific project tag to this task. Thanks!

We can get these same packages from osbpo.debian.net -- probably there's a bit of proxy config needed to make that work though.

Change #1272837 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack: use debian.net repo rather than the wmf-hosted repo

https://gerrit.wikimedia.org/r/1272837

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

pros

• Cloud server (re)images don't depend on osbpo.d.n being available
• No proxy changes needed

cons

• reprepro updates mechanism is not automated right now AFAIK, meaning package updates on osbpo require a reprepro checkupdate / reprepro update though in practice that might not be a problem (?)

In T423598#11837669, @fgiunchedi wrote:

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

How big is that component in total?

cons

• reprepro updates mechanism is not automated right now AFAIK, meaning package updates on osbpo require a reprepro checkupdate / reprepro update though in practice that might not be a problem (?)

I'd argue that's even rather a pro, since it allows us to control _when_ to sync in a controlled manner.

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

Fine w/me!

In T423598#11838641, @MoritzMuehlenhoff wrote:

In T423598#11837669, @fgiunchedi wrote:

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

How big is that component in total?

I took trixie-flaming-backports as an example, ~200MB across ~700 binary packages

$ curl https://osbpo.debian.net/osbpo/dists/trixie-flamingo-backports/main/binary-amd64/Packages -s | grep ^Size | awk '{print $2}' | numsum
186841276
$ curl https://osbpo.debian.net/osbpo/dists/trixie-flamingo-backports/main/binary-amd64/Packages -s | grep -c ^Package
669

cons

• reprepro updates mechanism is not automated right now AFAIK, meaning package updates on osbpo require a reprepro checkupdate / reprepro update though in practice that might not be a problem (?)

I'd argue that's even rather a pro, since it allows us to control _when_ to sync in a controlled manner.

indeed, in terms of being able to roll out upgrades it is a pro indeed

In T423598#11837669, @fgiunchedi wrote:

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

In practice this means following https://wikitech.wikimedia.org/wiki/Reprepro#Adding_a_new_external_repository

Filippo says: https://wikitech.wikimedia.org/wiki/Reprepro#Adding_a_new_external_repository should have everything

In T423598#11838816, @fgiunchedi wrote:

In T423598#11838641, @MoritzMuehlenhoff wrote:

In T423598#11837669, @fgiunchedi wrote:

I think we should be considering importing osbpo.debian.net apt repo as an upstream into aptrepo puppet module (i.e. modules/aptrepo/files/updates and related) and serve it locally from apt.w.o.

How big is that component in total?

I took trixie-flaming-backports as an example, ~200MB across ~700 binary packages

Ok, that's totally within reasonable limits for mirroring a component apt.wikimedia.org

Change #1276009 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add upstream repos for openstack flamingo and gazpacho

https://gerrit.wikimedia.org/r/1276009

Change #1276011 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Openstack: get osbpo packages from apt.wikimedia.org

https://gerrit.wikimedia.org/r/1276011

Change #1276009 merged by Andrew Bogott:

[operations/puppet@production] Add upstream repos for openstack flamingo and gazpacho

https://gerrit.wikimedia.org/r/1276009

Change #1278472 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):

[operations/puppet@production] aptrepo: add updates for trixie-wikimedia + osbpo

https://gerrit.wikimedia.org/r/1278472

Change #1278473 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Add openstack repos from debian.net to reprepro, take two

https://gerrit.wikimedia.org/r/1278473

Change #1278473 abandoned by Andrew Bogott:

[operations/puppet@production] Add openstack repos from debian.net to reprepro, take two

Reason:

dropping in favor of https://gerrit.wikimedia.org/r/c/operations/puppet/+/1278472

https://gerrit.wikimedia.org/r/1278473

Change #1278472 merged by Andrew Bogott:

[operations/puppet@production] aptrepo: add updates for trixie-wikimedia + osbpo

https://gerrit.wikimedia.org/r/1278472

Change #1276011 merged by Andrew Bogott:

[operations/puppet@production] Openstack: get osbpo packages from apt.wikimedia.org

https://gerrit.wikimedia.org/r/1276011

Change #1278543 had a related patch set uploaded (by Andrew Bogott; author: Andrew Bogott):

[operations/puppet@production] Fix keyfile for wikimedia openstack apt components

https://gerrit.wikimedia.org/r/1278543

Change #1278543 merged by Andrew Bogott:

[operations/puppet@production] Fix keyfile for wikimedia openstack apt components

https://gerrit.wikimedia.org/r/1278543

Change #1272837 abandoned by Andrew Bogott:

[operations/puppet@production] Openstack: use debian.net repo rather than the wmf-hosted repo

https://gerrit.wikimedia.org/r/1272837