Page MenuHomePhabricator

mwcli: Automate upload of new version to a releases server
Closed, ResolvedPublicFeature

Description

When a new version of mwcli is tagged, the artifact built in CI should be automatically copied to a releases server instead of done manually.

Event Timeline

So, do we actually think we need them to land on releases.wikimedia.org?
Perhaps we would be fine with releases from gitlab itself?

So, do we actually think we need them to land on releases.wikimedia.org?
Perhaps we would be fine with releases from gitlab itself?

It took me a minute to find the releases UI in GitLab: https://gitlab.wikimedia.org/releng/cli/-/releases (docs)

Publishing a release via the CI pipeline on GitLab sounds reasonable to me.

That sounds like something I'd be interested in and also will aim to look at as one of the next things, as it would allow be to fully retire this github repo that I have been side maintaining!

That sounds like something I'd be interested in and also will aim to look at as one of the next things, as it would allow be to fully retire this github repo that I have been side maintaining!

Maybe this is somewhat blocked on T286958: Document long-term requirements for GitLab job runners, though? See also https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner#Privacy_considerations

We also have to think about how to secure the artifacts and test results of jobs running in public clouds. How do we implement trust? How do we check if artifacts (images, compiled code) or test results weren't compromised?

cc @dduvall

That last quote you use refers to public clouds though.
The current runner for the mwcli is only used by mwcli, is part of the integration cloud project.
So I don't think there should be an issue with having some secret there that enables CI for the project to upload releases using the gitlab releases API.

Again the current alternative (currently used) is that this is all built (secrets and all) in github actions and published there.
And users of the package (at least now) are fine with that.

That last quote you use refers to public clouds though.
The current runner for the mwcli is only used by mwcli, is part of the integration cloud project.

What is the integration cloud project?

So I don't think there should be an issue with having some secret there that enables CI for the project to upload releases using the gitlab releases API.

Again the current alternative (currently used) is that this is all built (secrets and all) in github actions and published there.
And users of the package (at least now) are fine with that.

I think it's less of an issue of secrets, and more that building the artifact happens on a secure, audited / auditable server. Like, we want to avoid the situation where the runner is on a server that's compromised and then developers running mw {foo} are in fact exfiltrating their SSH keys to somewhere else.

I played around with releasing on gitlab today in my test repo

https://gitlab.wikimedia.org/addshore/test/-/releases
Made a release with artefacts automatically on tag
https://gitlab.wikimedia.org/addshore/test/-/releases/test-release-0015
Using the following configuration
https://gitlab.wikimedia.org/addshore/test/-/blob/main/.gitlab-ci.yml#L47-89

I'm going to work on finishing T288502: [mwcli] Archive Gerrit and Github repos. Long live Gitlab which requires moving the current dev releases from github to gitlab and rewriting the update code to work from gitlab :)

And if anyone ever stumbles on this ticket for making such releases, here is an updated example that will dynamicaly include all the files in the dir in a release

https://gitlab.wikimedia.org/addshore/test/-/blob/main/.gitlab-ci.yml#L47-90
https://gitlab.wikimedia.org/addshore/test/-/releases/test-release-0021

Addshore renamed this task from mwcli: Automate upload of new version to releases server to mwcli: Automate upload of new version to a releases server.Sep 20 2021, 7:47 PM
Addshore claimed this task.
Addshore moved this task from Backlog to In Progress on the mwcli board.
Addshore updated the task description. (Show Details)