Page MenuHomePhabricator

[builds-builder] Support adding repositories for Apt buildpack
Open, LowPublicFeature

Description

As a tool maintainer
I want to install packages from PPAs or other non-Canonical (pun intended) origins
So I can use software that is not present in the default repos.

The buildpack at https://gitlab.wikimedia.org/repos/cloud/toolforge/buildpacks/apt-buildpack had some version of this functionality until it was removed by e0136496: Explicitly disallow external packages/repos. That commit left a difficult to spot note in the builder output that users with specific needs for alternate apt repos file a task.

It is currently unclear what process would be used to adjudicate such a request, but it seems reasonable to assume that the bar for adoption would be set fairly high. Shared modifications of the buildpack for all users would exclude any use cases where a different origin was desired for a package whether that need be short term testing or long term use by a particular tool. Allowing container local modifications does in theory increase the attack surface of that container, but does not directly impact all other Aptfile buildpack users.

Event Timeline

One concrete use case for this feature would be installing toolforge-*-cli packages in a buildservice managed container. This is an idea that @Anomie and I have discussed as a potential partial solution for T356377: [toolforge] simplify calling the different toolforge apis from within the containers / T321919: Figure out and document how to call the Kubernetes API as your tool user from inside a pod.

Another would be installing packages from a tool maintainer managed PPA where things like custom builds of upstream projects could be staged for use. This could be a partial solution for T363028: Replace custom deployment with build service and job service.

bd808 renamed this task from [builds] Support adding repositories for Apt buildpack to [builds-builder] Support adding repositories for Apt buildpack.Apr 19 2024, 11:11 PM
dcaro triaged this task as Low priority.Jan 22 2025, 3:25 PM
dcaro subscribed.

I'll set this as low for now, until we have more use cases, there's more than just adding the toolforge repos to using the clis :/ (ubuntu vs debian...) so currently using pip/requirements.txt is the best way to get them (working on that for @Anomie's issue).

T380108: ircservserv not detecting message sender when running behind ZNC v1.8.2 is another concrete use case where adding a PPA would enable installing newer software versions than are in the currently available apt repos.

T380108: ircservserv not detecting message sender when running behind ZNC v1.8.2 is another concrete use case where adding a PPA would enable installing newer software versions than are in the currently available apt repos.

That one will also be sorted out with the upgrade to the ubuntu 24 runners though right?

T380108: ircservserv not detecting message sender when running behind ZNC v1.8.2 is another concrete use case where adding a PPA would enable installing newer software versions than are in the currently available apt repos.

That one will also be sorted out with the upgrade to the ubuntu 24 runners though right?

Yes, the particular problem of not having a new enough ZNC service available for installation could be solved multiple ways. In T380108#10329412 I imagined 3 concrete solutions: T380127: [builds-builder] Add support for Heroku's "24" builder stack based on Ubuntu 2024.04 noble, T363027: [builds-builder] Support adding repositories for Apt buildpack, or T363033: [builds-builder] Support using custom buildpacks. I actually think all three solutions would prove beneficial to the Toolforge community and not just the tools I am attempting to maintain.

I actually think all three solutions would prove beneficial to the Toolforge community and not just the tools I am attempting to maintain.

And that's great and appreciated, please keep bringing up ideas, though I hope you understand that not every idea can be implemented.

We still have heroku/deb-packages 0.1.3 when using --use-latest-versions. The custom repo support was added to heroku/deb-packages 0.2.0.

A docker pull docker.io/heroku/builder:24 today has heroku/deb-packages 0.2.0.