With a little work, the tools-package-builder-02 host in Toolforge could be set up with a simple and more modern workflow for building Debian packages.
Since the role is now role::wmcs::toolforge::package_builder, it should be possible to move to an sbuild based procedure with everything in place for users to build for the needed distros.
This will also include updating the docs to include release procedures, etc.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | aborrero | T249837 Revamp the build process for debian packages in Toolforge | |||
| Declined | None | T272289 wmcs-package-build.py: add support for testing the packages | |||
| Resolved | dcaro | T272290 wmcs-package-build.py: add support for creating a git tag |
I'm reviewing the current steps we have in our workflow:
Those are a lot of steps, but most of them are related to publishing and managing the apt repository and not building the package itself.
Building the package is a single step (step 5). If I introduce sbuild here we would still have a long workflow anyway.
I will work on replacing pdebuild with sbuild, but perhaps it makes sense to start a discussion on the whole workflow rather than the concrete build tool we use (which is just a little step in a long workflow).
What would be our ideal workflow here? Perhaps something like adding a tag in the git repo and then we get the deb package in the apt repo directly?
Change 587991 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] sbuild: introduce module and use it in toolforge package builder
Mentioned in SAL (#wikimedia-cloud) [2020-04-10T14:53:46Z] <arturo> live-hacking tools-puppetmaster-02 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/587991 for T249837
I think something like that would definitely be the ideal. I don't know how well-suited to that the pipeline config in production is. It may end up waiting on things like T249946: Basic evaluation of gitea as a repo setup? I could be wrong about that, though. It's just very focused around deploying to production k8s at this time vs something like committing into our repos.
Our own repo mirrors would definitely seem capable on the other hand. That's why I'm looking at the mirroring ability of things like that.
We could script some of the steps for distributing on the services nodes at least. That might not be a bad intermediate step.
Change 587991 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] sbuild: introduce module and use it in toolforge package builder
Change 588987 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] sbuild: chroot: update it using the root user
Change 588987 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] sbuild: chroot: update it using the root user
Change 591026 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: add wmcs-package-build.py script
Change 591026 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: add wmcs-package-build.py script
I introduced the wmcs-package-build.py script and created https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Packaging
comments and changes welcome :-)
Change 591318 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] toolforge: wmcs-package-build: don't use the $INSTANCEPROJECT env var
Change 591318 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] toolforge: wmcs-package-build: don't use the $INSTANCEPROJECT env var
Summary of what was done here:
I think I'll stop working on this for now, until we need more iterations.
I may have some ideas on facilitating even more the testing process etc by extending the script. For example, being able to handle the case in which we need a quick package release just for testing purposes (i.e, per-change package build so we are able to tests individual patches). This might be as easy as generating a temporal (snapshot) d/changelog entry, building the package and publishing a "testing" repo (or just toolsbe.
Please @Bstorm reopen if required.