Maniphest T320667

Cloud services enhancement proposal: Toolforge Kubernetes component workflow improvements
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	taavi
	Oct 12 2022, 5:54 PM

Description

Proposal Title: Toolforge Kubernetes component workflow improvements

Brief description: Proposal to partially replace the workflow used to deploy some Kubernetes components.

Why: see linked design doc on why the current workflow isn't ideal

Design documentation: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_Kubernetes_component_workflow_improvements

More info: this is a follow-up of the incident earlier this week when undeployed registry-admission changes started causing chaos

Related Objects
Search...

Status	Assigned	Task
Resolved	taavi	T320446 The following container images did not match any of the allowed registries ([['docker-registry.tools.wmflabs.org']])
Resolved	taavi	T320476 do not use :latest for toolforge infrastructure components
Resolved	taavi	T320667 Cloud services enhancement proposal: Toolforge Kubernetes component workflow improvements
Resolved	taavi	T329530 Convert all Toolforge custom components to standardized Helm based deployment
Resolved	dcaro	T336130 Automatically build Toolforge infrastructure container images in GitLab
Resolved	dcaro	T334399 Move Toolforge PipelineLib repositories to GitLab
Resolved	dcaro	T341084 [toolforge] Move all the components to the gitlab ci/cd flow
Open	None	T344765 Migrate striker to GitLab
Open	None	T334419 Set up translatewiki.net exports to push (and merge) to Wikimedia GitLab
Resolved	dcaro	T339198 Decision request - Toolforge component deployment flow details

Event Timeline

taavi created this task.Oct 12 2022, 5:54 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 12 2022, 5:54 PM

taavi added a parent task: T320476: do not use :latest for toolforge infrastructure components.Oct 12 2022, 5:55 PM

I agree with the overall proposal in the linked Wikitech page. I made a few comments to @taavi via IRC and we're iterating over it.

One thing I want to test is if gitlab.w.o provides an out-of-the-box helm chart rep (see https://docs.gitlab.com/ee/user/packages/helm_repository/)

I created https://gitlab.wikimedia.org/repos/cloud/toolforge/helm-charts to test the theory.

I was able to push to the helm chart repository here https://gitlab.wikimedia.org/repos/cloud/toolforge/helm-charts/-/packages

I'll let others decide if that's enough for what we want or, if harbor should be used, of whatever.

In T320667#8314522, @aborrero wrote:

I was able to push to the helm chart repository here https://gitlab.wikimedia.org/repos/cloud/toolforge/helm-charts/-/packages

The harbor equivalent on toolsbeta is:
https://harbor.toolsbeta.wmflabs.org/harbor/projects/4/helm-charts/registry-admission/versions/0.1.0

taavi added a project: Toolforge.Dec 6 2022, 1:37 PM

taavi moved this task from Inbox to Implementation on the Cloud Services Proposals board.Feb 13 2023, 4:09 PM

taavi mentioned this in T328539: toolforge: consider relocating core k8s components out of puppet into its own repository.Mar 6 2023, 1:03 PM

taavi mentioned this in T291915: toolforge: automate how we deploy custom k8s components.Mar 6 2023, 2:09 PM

taavi mentioned this in T295258: Automatically build Docker images for Toolforge custom components.

• nskaggs subscribed.Mar 14 2023, 2:58 PM

taavi mentioned this in T334399: Move Toolforge PipelineLib repositories to GitLab.Apr 10 2023, 2:29 PM

taavi added a subtask: T334399: Move Toolforge PipelineLib repositories to GitLab.Apr 10 2023, 2:32 PM

dcaro added a subtask: T339198: Decision request - Toolforge component deployment flow details.Jul 3 2023, 8:47 AM

dcaro closed subtask T336130: Automatically build Toolforge infrastructure container images in GitLab as Resolved.Jul 10 2023, 12:05 PM

taavi closed subtask T329530: Convert all Toolforge custom components to standardized Helm based deployment as Resolved.Sep 21 2023, 4:42 PM