Hi @Dzahn

I was looking at your work to migrate bugzilla to kubernetes with miscweb, and I am hoping you could help us determine the path forward for the microsites maintained by the research team.

• Is the summary in the description more or less correct?
• The parent task T332953 also contains a reference to miscweb. Is there a plan to migrate this to gitlab? Is there additional work required on the deployment side of things on the gitlab/kokkuri side?
• The section on how to switch microsites is relevant. However, I couldn't find (or maybe don't have access) to the referenced google doc, and we don't have puppet knowledge in our team.

Any recommendations/pointers you might have are much appreciated, thank you.

Hi @fkaelin yes, we also would like to move research.wikimedia.org to kubernetes (and the repo from Gerrit to Gitlab). This is basically a subtask of T300171.

Since we have already moved a couple other microsites lately and it's always following the same pattern, this isn't that big of a deal anymore to us. And your team doesn't have to know puppet for that. Moving to kubernetes means to stop using puppet and also we would do all that for you. Basically you don't have to worry about the actual move and @Jelto would be doing that.

There is one part though that needs discussion and is the reason we haven't done this until now. The question how exactly (and who) will deploy changes to the site in the future.

Right now some of you have +2 in the relevant Gerrit repo that hosts the content and you can submit and puppet will git pull from it to make the changes show up in production.

With a move to kubernetes though a change to the site will mean someone has to deploy a new version in kubernetes, via deployment servers.

For most services this is expected to be a self-service thing, so service-owners get deployment rights.

So either your team would get deployment rights and you could control it yourself (seems appropriate if you have changes quite often) or you would have to ask my team to deploy new versions (seems appropriate if you have changes less often).

Personally I am going on sabbatical today, so I would like to ask you to have a chat with Jelto and the rest of my team how to go about that.

Other than that, it shouldn't be a problem. Cheers!

@Dzahn Thank you kindly for this information. We will discuss regarding the deployment options and discuss with @Jelto regarding the migration itself.

And please enjoy the sabbatical!

Sounds good! Thank you!

Some minor corrections since you asked about the ticket content:

These are deployed via jenkins as microsites.

They are not deployed by jenkins. They are deployed by puppet. Specifically puppet code is using the git::clone class to pull from repositories on Gerrit.

This git::clone class can take a parameter for the source it clones from, it can be Gerrit OR Gitlab there meanwhile.

This means it's also possible to just change that and migrate your content repos from Gerrit to Gitlab first and treat the move to Kubernetes as a separate event.

Both are desired though and when moving to kubernetes we would also include migrating the repos to Gitlab.

(Miscweb1002).

It's meanwhile miscweb1003 and miscweb2003, since we upgraded to a newer OS version and new VMs.

the Ganeti infrastructure, which is being replaced with the kubernetes cluster.

Ganeti VMs will still exist for a while and have their own use cases that kubernetes can't cover (yet), but for these microsites / webservers, this is true and the goal, yea.

Hello. Do you have any updates on the migration of research and wikiworkshop to GitLab? Both sites look quite small and similar to the other sites we already migrated. So I don't see any blocker here. @Dzahn described the different deployment strategies quite well.

I have both sites in my tracking task T300171 to migrate all miscweb sites to GitLab and Kubernetes. So I can prepare/clone the GitLab project, CI and Kubernetes deployment similar to the existing sites. Then we can make sure you know how to deploy to Kubernetes. The steps I follow to migrate a miscweb site are documented here: https://wikitech.wikimedia.org/wiki/Miscweb/Kubernetes_migration_steps.

It should be fine if you keep updating the existing repositories in Gerrit in the meantime. We should be able to rebase that once we switched the service.

@Jelto a quick note that fkaelin is out of office until August 8th. He'll get back to you when he's back. (Thanks for the ping here and the help.)

Hi @Jelto, thanks for elaborating further and the links.

The team prefers to be able to deploy the pages ourselves, we can start with me (fab) and then I can add and show others on the team as necessary.

Regarding the naming: as the repos will be in the repos/sre group as opposed to research, using the name landing-page might be unclear. Should we change it to research-landing-page? That is not a problem for the wikiworkshop page.

The rate of change to these pages is low, if you let us know when you tackle this we should be able to avoid any rebasing due to changes during the migration.

jelto opened https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/40

add research miscweb sites to Trusted Runners

jelto merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/40

add research miscweb sites to Trusted Runners

jelto opened https://gitlab.wikimedia.org/repos/sre/miscweb/research-landing-page/-/merge_requests/1

add ci for research-landing-page image build

jelto merged https://gitlab.wikimedia.org/repos/sre/miscweb/research-landing-page/-/merge_requests/1

add ci for research-landing-page image build

jelto opened https://gitlab.wikimedia.org/repos/sre/miscweb/wikiworkshop/-/merge_requests/1

add ci for wikiworkshop image build

jelto merged https://gitlab.wikimedia.org/repos/sre/miscweb/wikiworkshop/-/merge_requests/1

add ci for wikiworkshop image build

Change 948539 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/deployment-charts@master] miscweb: add wikiworkshop and reasearch-landing-page to staging wikikube

https://gerrit.wikimedia.org/r/948539

Change 948539 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: add wikiworkshop and reasearch-landing-page to staging wikikube

https://gerrit.wikimedia.org/r/948539

Both services are migrated to GitLab (https://gitlab.wikimedia.org/repos/sre/miscweb/wikiworkshop and https://gitlab.wikimedia.org/repos/sre/miscweb/research-landing-page) and deployed to wikikube staging. Everything looks good so far.

jelto@deploy1002:/srv/deployment-charts/helmfile.d/services/miscweb$  curl -I --resolve research.wikimedia.org:30443:$(dig +short k8s-ingress-staging.discovery.wmnet) https://research.wikimedia.org:30443/
HTTP/2 200 
date: Tue, 15 Aug 2023 08:43:32 GMT
server: istio-envoy
last-modified: Mon, 14 Aug 2023 08:46:53 GMT
accept-ranges: bytes
content-length: 6902
content-type: text/html

jelto@deploy1002:/srv/deployment-charts/helmfile.d/services/miscweb$  curl -I --resolve wikiworkshop.org:30443:$(dig +short k8s-ingress-staging.discovery.wmnet) https://wikiworkshop.org:30443/
HTTP/2 200 
date: Tue, 15 Aug 2023 08:49:57 GMT
server: istio-envoy
last-modified: Mon, 14 Aug 2023 08:43:40 GMT
accept-ranges: bytes
content-length: 77
cache-control: max-age=3600, must-revalidate
content-type: text/html

I'll deploy the service to wikikube production soon. Then we can find a timeslot for the actual traffic switch from the miscweb vm to wikikube/Kubernetes and some onboarding (like how to deploy new version).

Change 948998 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/deployment-charts@master] miscweb: add wikiworkshop and reasearch-landing-page to eqiad and codfw

https://gerrit.wikimedia.org/r/948998

Great, thank you for the update @Jelto.

Change 948998 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: add wikiworkshop and reasearch-landing-page to eqiad and codfw

https://gerrit.wikimedia.org/r/948998

Change 949842 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/puppet@production] trafficserver: switch wikiworkshop.org and research.wikimedia.org

https://gerrit.wikimedia.org/r/949842

Change 949845 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/deployment-charts@master] miscweb: add www.wikiworkshop.org to extraFQDNs

https://gerrit.wikimedia.org/r/949845

Change 949845 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: add www.wikiworkshop.org to extraFQDNs

https://gerrit.wikimedia.org/r/949845

On the infrastructure side everything is prepared for switching both services to GitLab and Kubernetes/wikikube.

I invited @fkaelin to both GitLab projects https://gitlab.wikimedia.org/repos/sre/miscweb/wikiworkshop and https://gitlab.wikimedia.org/repos/sre/miscweb/research-landing-page with maintainer permissions. Furthermore @fkaelin is member of the deployment group and has permissions to deploy new versions.

I'd recommend to make you familiar and bookmark the new repos https://gitlab.wikimedia.org/repos/sre/miscweb/wikiworkshop and https://gitlab.wikimedia.org/repos/sre/miscweb/research-landing-page. The folder structure changed a bit, but most of your important files should be in html folder.

@fkaelin do you have experience deploying services to Kubernetes. The generic service deployment docs are here: https://wikitech.wikimedia.org/wiki/Kubernetes/Deployments, miscweb related docs here: https://wikitech.wikimedia.org/wiki/Miscweb#How_to_deploy_changes. But if you want we can have a short session and do one or two deployments together.

Do you have a preference when to switch to wikikube? That's mostly just merging https://gerrit.wikimedia.org/r/949845 and archiving the old repos.

Great, thank you @Jelto.

It would be great to have a short session do a deployment, I put something in your calendar.

As for the switch to wikikube, this can be done now - I added a +1 to the review.

Thanks you! Yes sure, that works. Let's do a short session tomorrow.

I'll deploy the switch to wikikube tomorrow UTC morning.

Can you verify that you are able to ssh to deploy1002.eqiad.wmnet?
I'm not sure if you used SSH before, if not, some docs are here: https://wikitech.wikimedia.org/wiki/SRE/Production_access#SSH_configuration.

@Jelto Yes I have access to the deployment machine. Thank you again, talk to you then.

Change 949842 merged by Jelto:

[operations/puppet@production] trafficserver: switch wikiworkshop.org and research.wikimedia.org

https://gerrit.wikimedia.org/r/949842

Onboarding to the new deployment workflow and repos has happened. @fkaelin feel free to close the task if you don't have any more questions.

If you have any problems make sure to use the collaboration-services tag to for any new tasks. Thanks again for the easy migration to Kubernetes!

I'll cleanup the obsolete references in the legacy miscweb puppet code.

Change 954069 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/puppet@production] miscweb/microsites: move monitoring of research pages to monitoring profile

https://gerrit.wikimedia.org/r/954069

Change 954070 had a related patch set uploaded (by Jelto; author: Jelto):

[operations/puppet@production] miscweb/microsites: remove wikiworkshop and research resources

https://gerrit.wikimedia.org/r/954070

Change 954069 merged by Jelto:

[operations/puppet@production] miscweb/microsites: move monitoring of research pages to monitoring profile

https://gerrit.wikimedia.org/r/954069

Change 954070 merged by Jelto:

[operations/puppet@production] miscweb/microsites: remove wikiworkshop and research resources

https://gerrit.wikimedia.org/r/954070