Page MenuHomePhabricator

Migrate mediawiki/services/ipoid to GitLab
Closed, ResolvedPublic

Description

Per the parent task:

Between implementing kokkuri and scaling our CI runners, we're ready to migrate any projects using our deployment pipeline from Gerrit to GitLab.

In this task, we can track the work to move https://gerrit.wikimedia.org/g/mediawiki/services/ipoid to GitLab.

- [ ] Make sure deployment-charts uses the image built by GitLab (T336163: Create helm chart for iPoid)

  • Remove Jenkins config related to PipelineLib (patch)
  • Archive the Gerrit repo (patch)
  • Mark the Gerrit repository read-only with a description pointing back to this task

Event Timeline

kostajh added subscribers: thcipriani, STran, Tchanders, sbassett.

From the parent task:

Archive your gerrit repo
Make you a GitLab repo
Add needed folks to the repo
Translate your .pipeline/config.yaml to work with GitLab

@thcipriani, who from Release-Engineering-Team could help us with this?

From the parent task:

Archive your gerrit repo
Make you a GitLab repo
Add needed folks to the repo
Translate your .pipeline/config.yaml to work with GitLab

@thcipriani, who from Release-Engineering-Team could help us with this?

Hey @kostajh we're taking a closer look at the mediawiki/services part of T335921: Migrate mediawiki/ namespace from Gerrit to GitLab this tomorrow. That looks like a prerequisite here. For now, I've added this task to our current sprint. I'll keep you updated.

kostajh updated the task description. (Show Details)
kostajh moved this task from Backlog to In Progress on the iPoid-Service board.

@Tchanders @STran With @Jelto's help, I set this up using the pipeline-to-gitlab conversion script. Comparing the test output in GitLab (https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/jobs/104100) to the existing one in Gerrit/Jenkins (https://integration.wikimedia.org/ci/job/ipoid-pipeline-test/18/console), I think this is working as intended.

I'm not sure if image building will just work, I think we'll need to merge the MR to check that.

I'm not sure if image building will just work, I think we'll need to merge the MR to check that.

The gitlab-ci.yml in your MR uses kokkuri, this should work and re-use the existing blubber file in .pipeline folder. To be able to build and push the image to our registry the project has to be allowed on the Trusted Runners. I created a MR above.

If that is merged, you should be able to merge your MR.

Hey @kostajh we're taking a closer look at the mediawiki/services part of T335921: Migrate mediawiki/ namespace from Gerrit to GitLab this tomorrow. That looks like a prerequisite here. For now, I've added this task to our current sprint. I'll keep you updated.

Thanks @thcipriani.

@kostajh has started setting up the Gitlab repo, and I'm wondering whether we should continue development on Gitlab and start archiving our gerrit repo now, or whether we should continue developing in gerrit until the prerequisite is solved. Might the blocking task delay our deployment if we switch to Gitlab now?

Merging this resulted in the GitLab building a production image and pushing to docker-registry.wikimedia.org, see this pipeline's output. I verified by running docker run --rm -it --entrypoint=/bin/cat docker-registry.wikimedia.org/repos/mediawiki/services/ipoid:latest .gitlab-ci.yml. (The file we added in the above merge request is present in the newly built image.)

One of the post-merge pipelines failed, related to publishing code coverage reports. We're discussing that here.

Otherwise, I think we have everything we need in order to use GitLab for code reviews and CI for the ipoid repository.

@Tchanders @STran is there anything else you'd like to see before we would switch to using GitLab for this repo?

Hey @kostajh we're taking a closer look at the mediawiki/services part of T335921: Migrate mediawiki/ namespace from Gerrit to GitLab this tomorrow. That looks like a prerequisite here. For now, I've added this task to our current sprint. I'll keep you updated.

Thanks @thcipriani.

@kostajh has started setting up the Gitlab repo, and I'm wondering whether we should continue development on Gitlab and start archiving our gerrit repo now, or whether we should continue developing in gerrit until the prerequisite is solved. Might the blocking task delay our deployment if we switch to Gitlab now?

Otherwise, I think we have everything we need in order to use GitLab for code reviews and CI for the ipoid repository.

I was just talking this through with some folks.

I think the only risk you run using GitLab is we're still working through T335921, so we're adjusting permissions in this space. That may be disruptive in ways we don't anticipate (we'll be mindful you're here, but worst case we lock you out and you yell at us to fix it). But I don't expect major disruptions apart from that: data is all safe and backed up, kokkuri-based workflows are unlikely to make dramatic changes. I think you're safe to continue work on GitLab (with the above caveats).

EDIT: if the above sounded ominous, it wasn't meant to—please use GitLab :)

Change 926424 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[operations/deployment-charts@master] ipoid: Set sources to point to GitLab

https://gerrit.wikimedia.org/r/926424

Change 927125 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/config@master] jjb: Remove ipoid config

https://gerrit.wikimedia.org/r/927125

Change 927124 had a related patch set uploaded (by Hashar; author: Kosta Harlan):

[mediawiki/services/ipoid@master] Empty repository, update README to point to GitLab

https://gerrit.wikimedia.org/r/927124

Change 927125 merged by jenkins-bot:

[integration/config@master] jjb: Remove ipoid config

https://gerrit.wikimedia.org/r/927125

Change 927124 merged by Hashar:

[mediawiki/services/ipoid@master] Empty repository, update README to point to GitLab

https://gerrit.wikimedia.org/r/927124

hashar updated the task description. (Show Details)
hashar moved this task from Untriaged to Archive tools & others on the Projects-Cleanup board.
hashar added a subscriber: hashar.

I have merged Kosta patch to empty up the Gerrit repository and marked it Read-Only https://gerrit.wikimedia.org/r/admin/repos/mediawiki/services/ipoid,general

Change 926424 merged by jenkins-bot:

[operations/deployment-charts@master] ipoid: Update for GitLab migration

https://gerrit.wikimedia.org/r/926424

As this project migrates over to Gitlab, the new security pipeline templates can be used. I think that https://www.mediawiki.org/wiki/Security/Application_Security_Pipeline#Npm_Audit would be the best fit. I can open a merge request in gitlab for this as well if that is preferred.

As this project migrates over to Gitlab, the new security pipeline templates can be used. I think that https://www.mediawiki.org/wiki/Security/Application_Security_Pipeline#Npm_Audit would be the best fit. I can open a merge request in gitlab for this as well if that is preferred.

@Mstyles a merge request (and separate phab task to track it) would be nice, thank you!

kostajh updated the task description. (Show Details)

- [ ] Make sure deployment-charts uses the image built by GitLab (T336163: Create helm chart for iPoid)

See also T338224: Provide ability to tag GitLab CI built images with a datetime format, set as default in pipeline-to-gitlab conversion; in the meantime, one can use tags from https://docker-registry.wikimedia.org/repos/mediawiki/services/ipoid/tags/. serviceops also wants to clarify where we are publishing those images, but that doesn't need to happen in this

So, I think we can mark this task as resolved.

These are just the standard node16 images as defined by https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/blob/main/.pipeline/blubber.yaml, correct? Nothing that can quite be used as a dev/CI image with mariadb support, or at least not yet.

These are just the standard node16 images as defined by https://gitlab.wikimedia.org/repos/mediawiki/services/ipoid/-/blob/main/.pipeline/blubber.yaml, correct? Nothing that can quite be used as a dev/CI image with mariadb support, or at least not yet.

Right. Per T308789#8891259 I think we will use the services: feature in GitLab CI to run a MariaDB container alongside the node container, if/when we add integration tests that would use this functionality.