Page MenuHomePhabricator

Locate an alternative source for modern Openstack Debian packages
Closed, ResolvedPublic


Several services in Labs, tools, and prod rely on packages from the mirantis apt repo. That repo doesn't work, so those systems currently cannot be rebuilt.

Event Timeline

One bandaid we could try would be to gather up packages from /var/cache/apt/archives/ on our hosts and put them into an aptly repo. This is obviously a very temporary solution, but arguable better than not being able to rebuild a box if it goes down.

Jessie-backports currently has Mitaka package versions. As noted by Mortiz on irc it is unlikely that those packages will be updated again now that stretch is released. There is a bit of scariness in not having an easy way to pin our installed packages to the mitaka versions like the Mirantis repos did, but its probably something we can survive.

Change 369957 had a related patch set uploaded (by Rush; owner: cpettet):
[operations/puppet@production] openstack: Jessie has Liberty py3 client libraries

Change 369957 merged by Rush:
[operations/puppet@production] openstack: Jessie has Liberty py3 client libraries

Change 369972 had a related patch set uploaded (by Rush; owner: cpettet):
[operations/puppet@production] openstack: Jessie has cannot fulfill py3 clients without backports

Change 369972 merged by Rush:
[operations/puppet@production] openstack: Jessie has cannot fulfill py3 clients without backports

Change 369987 had a related patch set uploaded (by Rush; owner: cpettet):
[operations/puppet@production] openstack: revert shenkengen to py2 libs

Change 369987 merged by Madhuvishy:
[operations/puppet@production] openstack: revert shenkengen to py2 libs

In looking at a total move to Debian as solution there is at least one glaring issue in that the cadence of stable=>backports(testing) from next release does not seem to follow a N+1 release cycle. For instance Jessie=>Stretch=>Buster is poised to miss Ocata entirely. That's a pretty significant hurdle if it's sane for us at all.

Notes from investigating:

OpenStack Upgrade release policy states

We only test N to N+1 in the CI system (the grenade jobs) so while
things might work in practice by skipping a release, it's not tested
 that way upstream at all so any chance of it working is accidental and
shouldn't be relied upon or really expected to be supported that way upstream.

A the moment it sure looks like there is no way to avoid a troubling and expected faulty version jump from jessie=>stretch=>buster (ocata is missing) <--- package version matrix <--snapshot for keystone

Openstack release(s) by codename

v9 == mitaka
v10 == newton
v11 == ocata
v12 == pike

Jessie Stable: (Icehouse)
jessie (oldstable) (python): OpenStack identity service
2014.1.3-6: all

Jessie Backports: (Mitaka)
jessie-backports (python): OpenStack identity service
2:9.0.0-2~bpo8+1: all

Debian 9 “Stretch” LTS is targeted at 2020 to June 2022

Stretch Stable: (Newton)
stretch (stable) (python): OpenStack identity service
2:10.0.0-9: all

Stretch Backports:

  • None

Buster Stable:(Pike)
buster (testing) (python): OpenStack identity service
2:12.0.0-2: all
Buster Backports:

I've been assuming that we need to move to source-based deployments since Debian seems determined to do this the wrong way. I've contacted the debian packager (Allison Randal) several times about this but haven't gotten any useful responses.

Skipping versions seems semi-possible if we have to, but given the current cadence I don't have a lot of faith in how they'll handle this going forward.

I can add a bit more details.


  • Ocata (v11) was not packaged for Debian on purpose
  • There are 2 options for using Ocata in Debian: build ourselves the packages OR use the repo

13:25 <arturo> DebianDeveloper: did you read my message regarding keystone v10 --> v11 --> v12?
13:26 <DebianDeveloper> arturo: Not sure, can you say again?
13:27 <arturo> sure
13:27 <arturo> DebianDeveloper: <--- there seem to be no upgrade path for keystone in debian, right?
13:27 <DebianDeveloper> arturo: I skipped packaging Ocata, indeed, but we do have the source code available in the Git.
13:28 <DebianDeveloper> So you "just" need to rebuild it from the Git.
13:28 <arturo> I see
13:28 <arturo> was there any technical reason for the skip?
13:28 <DebianDeveloper> There's also a repository with backports for Ocata in Stretch done by the CZ people.
13:29 <arturo> oh, could you share the link?
13:29 <DebianDeveloper> arturo: There's a social one: I got out of employer, and was a bit demotivated, plus I expected others would pick-up, which never happened.
13:29 <arturo> DebianDeveloper: ok, understood
13:30 <DebianDeveloper> arturo: repo : deb debian-stretch ocata ocata-backports
13:30 <DebianDeveloper> apt_key :
13:30 <arturo> thanks!
13:31 <DebianDeveloper> arturo: I thought about publishing that on a repo, but so far, I didn't.
13:31 <arturo> that would be great, indeed
13:32 <DebianDeveloper> arturo: If I was to publish Pike and Queens on official Debian backports, would you use it?
13:32 <DebianDeveloper> That's about 3/4 days of work, which is why I don't want to do it for nothing.
13:33 <arturo> DebianDeveloper: I fully understand the think about sharing efforts et al
13:34 <arturo> DebianDeveloper: about Pike/Queens, we are currently evaluating in the team how to do new deployments
13:34 <arturo> I bet for using Debian, that's for sure, but others in my team prefer source deployments
13:35 <arturo> (one of the points being the upgrade path thing)
13:39 <DebianDeveloper> arturo: If using sources, it means you're re-inventing all what's in the package maintainer's scripts. That's a huge amount of work.
13:40 <arturo> I see
13:42 <arturo> from the resources point of view, I guess there is little we could do from my team, given we are a non-profit foundation, fully donation-based income. Our resources are very limited. I wonder what happens with the big company players out there

I've checked the repo by hand and it's full of packages, apparently ready to use.

This is pretty much resolved thanks to various gymnastics with upstream debian repos. We're currently using jessie-backports for Mitaka.

@chasemp or @aborrero could you note here what repos we plan to use for the next few OpenStack releases? Then this can be closed.

According to my sources, Openstack Newton (mitaka + 1) is in Debian stretch standard repos. All releases after Newton (ocata, etc) are probably located in stretch-backports

This should contain ocata:

deb debian-stretch ocata
aborrero claimed this task.

So, conclusions:

  • openstack mitaka is in jessie-backports (already using it in eqiad1)
  • openstack newton is in stretch (mitaka+1)
  • openstack ocata is in stretch-backports (newton+1)

Closing task now.

If you are looking at this bug report, is probably because you are looking for this information:

The matrix suggests that Ocata packages are available in stretch-backports. I don't see that, though... were they removed, or am I looking in the wrong place?

root@cloudcontrol1003:~# apt-get update
Ign:1 stretch InRelease
Hit:2 stretch-updates InRelease
Get:3 stretch-backports InRelease [91.8 kB]
Hit:4 stretch-wikimedia InRelease
Hit:6 stretch Release                             
Hit:5 stretch/updates InRelease
Fetched 91.8 kB in 1s (55.2 kB/s)
No directory, logging in with HOME=/
INFO:debmonitor:Found 949 installed binary packages
INFO:debmonitor:Found 7 upgradable binary packages (including new dependencies)
INFO:debmonitor:Successfully sent the upgradable update to the DebMonitor server
Reading package lists... Done
root@cloudcontrol1003:~# apt-cache madison nova-compute
nova-compute | 2:14.0.0-4+deb9u1 | stretch/main amd64 Packages
nova-compute | 2:14.0.0-4+deb9u1 | stretch/updates/main amd64 Packages
      nova | 2:14.0.0-4+deb9u1 | stretch/main Sources
      nova | 2:14.0.0-4+deb9u1 | stretch/updates/main Sources