Page MenuHomePhabricator
Create Task
Maniphest T341084

[toolforge] Move all the components to the gitlab ci/cd flow
Closed, ResolvedPublic
Actions

Description

This might include moving to gitlab + moving to helmfile.

If the component is going to take considerable work, I'll open a subtask for it.

Components:

Details

SubjectRepoBranchLines +/-
MOVED TO GITLABlabs/tools/maintain-kubeusersmaster+9 -12 K
jobs-framework-emailer: move to gitlabcloud/toolforge/jobs-framework-emailermain+1 -2 K
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
dcaro updated the task description. (Show Details)Jul 11 2023, 9:41 AM
CodeReviewBot added a comment.Jul 11 2023, 10:02 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/1

deploy: use toolforge cd flows

Stashbot added a comment.Jul 11 2023, 10:14 AM

Mentioned in SAL (#wikimedia-cloud) [2023-07-11T10:14:02Z] <dcaro> deploy ingress-admission 0.0.38, ci rebuild (T341084)

CodeReviewBot added a comment.Jul 11 2023, 10:16 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/57

ingress-admission: add component

dcaro updated the task description. (Show Details)Jul 11 2023, 10:16 AM
CodeReviewBot added a comment.Jul 12 2023, 9:38 AM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/59

api-gateway: bump to 0.0.16

CodeReviewBot added a comment.Jul 12 2023, 9:45 AM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/59

api-gateway: bump to 0.0.16

aborrero moved this task from Next to Doing on the User-aborrero board.Jul 12 2023, 9:47 AM
Stashbot added a comment.Jul 12 2023, 10:06 AM

Mentioned in SAL (#wikimedia-cloud) [2023-07-12T10:06:42Z] <dcaro> deployed api-gateway 0.0.16, no changes, ci rebuild (T341084)

dcaro updated the task description. (Show Details)Jul 12 2023, 10:07 AM
CodeReviewBot added a comment.Jul 12 2023, 12:19 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/62

builds-admission: bump to 0.0.63

Stashbot added a comment.Jul 12 2023, 12:46 PM

Mentioned in SAL (#wikimedia-cloud) [2023-07-12T12:46:39Z] <arturo> deployed builds-admission 0.0.63-20230712120152-2ef80a7c (T341084)

Stashbot added a comment.Jul 12 2023, 12:46 PM

Mentioned in SAL (#wikimedia-cloud) [2023-07-12T12:46:46Z] <arturo> deployed builds-admission 0.0.63-20230712120152-2ef80a7c (T341084)

CodeReviewBot added a comment.Jul 12 2023, 12:49 PM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/62

builds-admission: bump to 0.0.63

dcaro updated the task description. (Show Details)Jul 12 2023, 12:49 PM
CodeReviewBot added a comment.Jul 12 2023, 2:13 PM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/cert-manager/-/merge_requests/3

Move to toolforge-deploy

dcaro added a comment.Jul 12 2023, 4:17 PM

I'll be taking a bit of a break on this until I'm back on pto, if anyone wants to keep at it while I'm away I'll be more than happy, feel free :)

dcaro updated the task description. (Show Details)Jul 13 2023, 1:53 PM
dcaro added a parent task: T334399: Move Toolforge PipelineLib repositories to GitLab.Jul 14 2023, 6:29 AM
dcaro mentioned this in T332953: Migrate PipelineLib repos to GitLab.Jul 14 2023, 8:09 AM
CodeReviewBot added a comment.Jul 14 2023, 3:01 PM

aborrero opened https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-emailer/-/merge_requests/4

jobs-framework-emailer: migrate to modern workflow

aborrero triaged this task as Medium priority.Jul 17 2023, 12:21 PM
CodeReviewBot added a comment.Jul 17 2023, 1:15 PM

aborrero opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/64

components: add jobs-emailer

CodeReviewBot added a comment.Jul 18 2023, 10:33 AM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/jobs-framework-emailer/-/merge_requests/4

jobs-framework-emailer: migrate to modern workflow

Stashbot added a comment.Jul 18 2023, 10:45 AM

Mentioned in SAL (#wikimedia-cloud) [2023-07-18T10:45:50Z] <arturo> redeploy jobs-emailer into k8s (T341084)

Stashbot added a comment.Jul 18 2023, 11:02 AM

Mentioned in SAL (#wikimedia-cloud) [2023-07-18T11:02:20Z] <arturo> redeploy jobs-emailer 0.0.41-20230718103342-3dddcfb8 into k8s (T341084)

CodeReviewBot added a comment.Jul 18 2023, 11:08 AM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/64

components: add jobs-emailer

aborrero updated the task description. (Show Details)Jul 18 2023, 11:09 AM
aborrero moved this task from Doing to Radar/observer on the User-aborrero board.Jul 18 2023, 12:03 PM
fnegri added projects: cloud-services-team (FY2023/2024-Q1-Q2), Goal.Jul 26 2023, 3:57 PM
fnegri moved this task from Backlog to Planned Goals on the cloud-services-team (FY2023/2024-Q1-Q2) board.
CodeReviewBot added a comment.Aug 2 2023, 9:23 AM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/68

registry-admission: add component

dcaro updated the task description. (Show Details)Aug 2 2023, 9:28 AM
dcaro updated the task description. (Show Details)
dcaro updated the task description. (Show Details)
dcaro updated the task description. (Show Details)Aug 2 2023, 9:31 AM
gerritbot added a comment.Aug 4 2023, 8:34 AM

Change 945753 had a related patch set uploaded (by David Caro; author: David Caro):

[labs/tools/maintain-kubeusers@master] MOVED TO GITLAB

https://gerrit.wikimedia.org/r/945753

CodeReviewBot added a comment.Aug 30 2023, 4:24 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/1

deploy: move to toolforge-deploy

CodeReviewBot added a comment.Sep 5 2023, 12:28 PM

dcaro updated https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/80

maintain_kubeusers: add component

gerritbot added a comment.Sep 5 2023, 12:44 PM

Change 945753 merged by jenkins-bot:

[labs/tools/maintain-kubeusers@master] MOVED TO GITLAB

https://gerrit.wikimedia.org/r/945753

CodeReviewBot added a comment.Sep 5 2023, 1:34 PM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/68

registry-admission: add component

dcaro updated the task description. (Show Details)Sep 5 2023, 1:35 PM
dcaro mentioned this in rLTMK2993a16dde66: MOVED TO GITLAB.Sep 5 2023, 1:56 PM
dcaro updated the task description. (Show Details)Sep 8 2023, 1:07 PM
dcaro added a parent task: T327057: [toolforge] repositories move to gitlab.
CodeReviewBot added a comment.Sep 8 2023, 2:05 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/buildservice/-/merge_requests/9

deploy: move to helmfile

CodeReviewBot added a comment.Sep 8 2023, 2:29 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/81

builds-builder: add component

CodeReviewBot added a comment.Sep 11 2023, 10:46 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/wmcs-k8s-metrics/-/merge_requests/3

deploy: move to the toolforge-deploy repository

Stashbot added a comment.Sep 11 2023, 11:36 AM

Mentioned in SAL (#wikimedia-cloud) [2023-09-11T11:36:29Z] <dcaro> deploy kubernetes-metrics (T341084)

dcaro added a comment.Sep 11 2023, 12:24 PM

For kubernetes-metrics, I had to manually update the helm manifest to match the existing resources.

I think this happened when we upgraded to k8s 1.22, where clusterrole and clusterrolebinding moved from v1beta1 to v1, we might have done it by hand and helm did not see the change.

Anyhow, the procedure for that is:

  • Download the existing manifest:
root@toolsbeta-test-k8s-control-5:/srv/git/toolforge-deploy# kubectl get secret -o json -n metrics sh.helm.release.v1.wmcs-metrics.v1 | jq '.data.release' -r | base64 -d | base64 -d | gunzip > metrics.helm-manifest

# edit the manifest and change rbac.authorization.k8s.io/v1beta1 -> rbac.authorization.k8s.io/v1

root@toolsbeta-test-k8s-control-5:/srv/git/toolforge-deploy# DATA=$(cat metrics.helm-manifest  | gzip -c | base64 --wrap=0| base64 --wrap=0)
root@toolsbeta-test-k8s-control-5:/srv/git/toolforge-deploy# kubectl patch secret -n metrics sh.helm.release.v1.wmcs-metrics.v1 --type='json' -p="[{\"op\":\"replace\",\"path\":\"/data/release\",\"value\":\"$DATA\"}]"
Stashbot added a comment.Sep 11 2023, 12:34 PM

Mentioned in SAL (#wikimedia-cloud) [2023-09-11T12:34:27Z] <dcaro> deploy kubernetes-metrics (T341084)

CodeReviewBot added a comment.Sep 11 2023, 12:36 PM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/53

kubernetes-metrics: add component

dcaro updated the task description. (Show Details)Sep 11 2023, 1:58 PM
Stashbot added a comment.Sep 11 2023, 4:05 PM

Mentioned in SAL (#wikimedia-cloud) [2023-09-11T16:05:17Z] <dcaro> deploy builds-builder (T341084)

CodeReviewBot added a comment.Sep 12 2023, 3:02 PM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/maintain-kubeusers/-/merge_requests/1

deploy: move to toolforge-deploy

Stashbot added a comment.Sep 13 2023, 10:27 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:27:31Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:27 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:27:56Z] <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:29 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:29:24Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:30 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:30:25Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:34 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:34:32Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:35 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:35:20Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:38 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:38:18Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:38 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:38:44Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:41 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:41:34Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 10:41 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T10:41:42Z] <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:37 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:37:08Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:37 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:37:23Z] <wm-bot2> dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:37 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:37:33Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:38 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:38:24Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:40 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:40:20Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:40 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:40:23Z] <wm-bot2> dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:41 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:41:07Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:41 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:41:09Z] <wm-bot2> dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:51 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:51:07Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 12:51 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T12:51:30Z] <wm-bot2> dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 4:40 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T16:40:25Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 4:41 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T16:41:10Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 5:13 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T17:13:16Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)

Stashbot added a comment.Sep 13 2023, 5:14 PM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-13T17:14:03Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)

CodeReviewBot added a comment.Sep 13 2023, 5:15 PM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/80

maintain_kubeusers: add component

Stashbot added a comment.Sep 14 2023, 8:09 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-14T08:09:33Z] <wm-bot2> dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone (T341084)

Stashbot added a comment.Sep 14 2023, 8:10 AM

Mentioned in SAL (#wikimedia-cloud-feed) [2023-09-14T08:10:18Z] <wm-bot2> dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admissionNone (T341084)

dcaro updated the task description. (Show Details)Sep 15 2023, 3:20 PM
CodeReviewBot added a comment.Sep 21 2023, 12:14 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/2

deploy: use toolforge cd flows

CodeReviewBot added a comment.Sep 21 2023, 12:14 PM

dcaro closed https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-admission/-/merge_requests/2

deploy: use toolforge cd flows

dcaro updated the task description. (Show Details)Sep 21 2023, 2:46 PM
dcaro updated the task description. (Show Details)
CodeReviewBot added a comment.Sep 21 2023, 3:53 PM

dcaro opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/96

README: update the docs enumerating the components

dcaro updated the task description. (Show Details)Sep 21 2023, 3:56 PM
dcaro moved this task from In Progress to Toolforge iteration 00 on the Toolforge board.Sep 22 2023, 8:52 AM
dcaro edited projects, added Toolforge (Toolforge iteration 00); removed Toolforge.
CodeReviewBot added a comment.Sep 25 2023, 8:57 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/builds-builder/-/merge_requests/9

deploy: move to helmfile

CodeReviewBot added a comment.Sep 25 2023, 9:56 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/81

builds-builder: add component

dcaro changed the task status from Open to In Progress.Sep 26 2023, 12:34 PM
dcaro moved this task from Next Up to In Progress on the Toolforge (Toolforge iteration 00) board.
CodeReviewBot added a comment.Oct 3 2023, 8:44 AM

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/96

README: update the docs enumerating the components

dcaro closed this task as Resolved.Oct 4 2023, 7:25 AM
dcaro moved this task from In Progress to Done on the Toolforge (Toolforge iteration 00) board.
fnegri moved this task from Planned Goals to Done on the cloud-services-team (FY2023/2024-Q1-Q2) board.Oct 5 2023, 4:35 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits