Page MenuHomePhabricator

[builds-builder] Add support for specifying multiple buildpacks to run against a repo
Open, Needs TriagePublicFeature

Description

As a tool maintainer
I want to create container images incorporating the output of multiple language runtime specific buildpacks
So I can operate tools which need for example golang and python components in the same container in order to do their work.

[14:53]  <    bd808> dcaro: hopefully quick question. With the stack we use today is it possible for both the python and golang buildpacks to run for the same build? Or is it only nodejs that can run along side another language buildpack?
[14:54]  <    dcaro> bd808: only specific ones yes, like nodejs/apt/procfile
[14:55]  <    dcaro> it should not be very hard to allow specifying which ones you want though of the ones available for power users
[14:55]  <    dcaro> (as we generate the list of "stacks" on the fly already)
[14:57]  <    bd808> that could be neat. I was trying to make a thing for the bridgebot tool with python + golang. Mostly because I had code I could copy-n-paste for most of the python bit.
[14:57]  <    dcaro> I think that there's no task for it yet though, feel free to open one, it's not in the current roadmap, but I might give it a go if I have an idle day/want to switch topics
[14:58]  <    bd808> For this particular project I need a thing in the container to generate a config file at runtime because the upstream code doesn't do 12-factor envvar stuff.
[14:59]  <    dcaro> xd
[14:59] bd808 will write a feature request later

Event Timeline

As explained briefly in the irc snippet, I have a use case where Tool-bridgebot uses a 3rd party golang component (matterbridge) which expects to read configuration from a static file. I want to move this tool from a legacy hand built deployment to a modern stack of a toolforge jobs managed deployment using a toolforge build generated image. To do so I need to replace the current chmod 0600 config file in the tool's $HOME with configuration that uses toolforge envvars for transmitting secrets into the running container.

I can do this by adding a launcher script into the generated image that interpolates envvars into a template config file prior to starting the matterbridge process. This is actually a pattern that is already in use by the Wikibugs tool for its ZNC bouncer component. In that project I used Python to write the launcher script, largely because I am very comfortable using Python for such things. I tried to copy the same pattern for bridgebot, but ran into this missing feature of being able to combine both the golang builder (to build a patched version of the upstream matterbridge project) and the python builder (to build the launcher script). Currently the python builder is detected and used with the golang builder never getting a chance to perform its normal workflow.

For this specific case I can find other workarounds, likely I will port my launcher to golang, but it feels like it would be generally nice to be able to perform this sort of mix and match image building if it is reasonably possible to implement on the backend.