I have a couple of hand-made jessie instances running now. Unfortunately, the software that we use to build official labs images (python-vm-builder) doesn't support Debian. There's a debian plugin that was last modified in 2009 and only offers to build Lenny... I'm currently updating/rewriting/rehabilitating that plugin to build modern Debian versions.

It's turning into quite a job.

In T75592#793510, @Andrew wrote:

Unfortunately, the software that we use to build official labs images (python-vm-builder) doesn't support Debian.

ahh.. i see. thanks for the explanation. so i guess the following is what you mentioned as last being modified in 2009, right:

https://wiki.debian.org/VMBuilder
git clone git://git.debian.org/git/pkg-escience/vmbuilder.git

This is now mostly working -- I've committed some patches to bootstrap-vz and built a custom package. Marc is currently working on lvm vs. systemd, it seems that many Jessie users are running into similar problems.

@coren @Andrew What's still blocking this? I heard there's some sort of upstream issue? Link?

The upstream bug that @coren referred to before is https://bugs.debian.org/616689 which affects us because /var is on LVM on Labs images. This isn't really blocking this, though; it can be easily worked around by passing rootdelay=N (which was the default in prod for Ubuntu as well, until 3d3868aa17b8670e847332b37ce8bd44cb530248) or possibly scsi_mod.scan=sync.

Moreover, I've expressed a very strong preference of ditching the separate /var from Labs images entirely (both Debian & newer Ubuntu) as it's an unnecessary delta from production and also, in my opinion, both useless & error-prone. This change alone would work around the upstream bug without the aforementioned workarounds. @coren has expressed a preference for doing this only if we move / to LVM as well so that /var can be expanded, which I don't mind, but I don't think it's strictly necessary either.

In any case, we aren't really blocked on anyone but us, as far as I know. This update is more than a week old, though, so I may have missed something.

I *really* think we should just move / to LVM and get it over with. The reason /var is on LVM is because disks on labs are usually tiny compared to production, and the default /var is only 2G (which is also terrible, I think). We've historically had to fight a lot of 'damn, /var/log is full now, grrrrrrr' issues at various times of the day, and I really hope we don't have to continue doing that.

So either we should generously increase /var from the currently-very-paltry 2G (to, idk, 10G?) or move everything to LVM so re-adjustment is flexible. I far prefer the latter.

This isn't really blocking this, though; it can be easily worked around by passing rootdelay=N (which was the default
in prod for Ubuntu as well, until 3d3868aa17b8670e847332b37ce8bd44cb530248) or possibly
scsi_mod.scan=sync.

I'm unable to get any joy from these suggestions. I have an instance now that sets rootdelay=300 and scsi_mod.scan=sync; it still drops into emergency mode and fails to boot.

OK, we've abandoned our partition scheme and agreed to make just one 20g / by default.

There's now an image live on labs, debian-8.0-jessie (testing). If it stands the test of time I'll remove the (testing) and call it official.

Few issues persist, have filed blocking tasks.

What's left to do here?