Page MenuHomePhabricator

Investigation: How to do continuous delivery from the release builds [Timebox: 2 hours]
Closed, ResolvedPublic

Event Timeline

toan renamed this task from Investigation: How to do continuous integration from the release builds [Timebox: 2 hours] to Investigation: How to do continuous delivery from the release builds [Timebox: 2 hours].Mar 12 2021, 10:59 AM

Ok, so to summarize what was attempted.

The idea would be to publish our docker-images to a separate account (or just separate tags) which could then easily be pulled and tested using the test-system. There is an attempted PR up here but because of the timebox i didn't quite reach anything that actually works.

Only publish if there are some changes
I tried using our build_metadata_* files to detect any changes that should be published using this script which seems to give the correct behaviour. HOWEVER the implementation of this (The caching of last build_metadata. Might also be able to get this from the API aswell) might have to depend on which route we would choose from the following bulletpoints.

Publish at the end of build
I tried using the our current publish/dockerhub.sh script at the end of the build step but this will not work out-of-the-box because the artifacts aren't present/published until the whole workflow run is completed.
To use this approach we would need to adapt publish/dockerhub.sh to make the downloading optional and instead fetch the artifacts from the current workflow run (the same way it's done in testing)

Queue a publish workflow run at the end of testing
This seems like it could work (atleast I got it working locally). By using a script like this we could queue a publish job with the correct parameters at the very end of the tests are done. This way we can still rely on what is already in the repo and nicely decouple the building with the actual publishing. HOWEVER since the pipeline currently is broken and im out of time for the investigation I wasn't able to verify that this would work inside the pipeline, so there is a risk that queing a workflow from within another workflow that depends on the original one might just not work (I haven't found any mentions of this being supported).

I am going to boldly call this done. The time allocated has been spent, and some initial insights have been generated.

Using separate images for this continuous deployment sounds sensible to me. Tag or something else would be implementation detail to figure out when this is done.
Publishing at the end of the build/post-testing also seems fine. I'd naively think that we'd like to treat the test system deployment as a separate step, which I believe is considered as well.
Sounds like there is some plan!

All in all, I think as long as there is a functioning manual way of updating the test system on demand, the continuous deployment topic could be de-prioritized for the time being - there are other pressing things to solve.