Page MenuHomePhabricator
Create Task
Maniphest T340182

move micro tendril.wikimedia.org and dbtree.wikimedia.org to kubernetes
Closed, ResolvedPublic
Actions

Description

Tracking migration of all microsites in T300171 gets a bit confusing. So opening a subtask for tendril/dbtree microsite.

Migration checklist (see complete steps here):

Details

SubjectRepoBranchLines +/-
microsites: add monitoring for statictendril site for retired servicesoperations/puppetproduction+24 -0
trafficserver: switch dbtree/tendril to k8s backendoperations/puppetproduction+3 -3
miscweb: add statictendril to eqiad and codfw k8soperations/deployment-chartsmaster+2 -2
miscweb: add statictendril release to miscweb stagingoperations/deployment-chartsmaster+23 -1
Customize query in gerrit
TitleReferenceAuthorSource BranchDest Branch
use timestamp as image tagrepos/sre/miscweb/statictendril!3jeltotimestamp-build-tagmain
fix image build for statictendrilrepos/sre/miscweb/statictendril!2jeltoblubber-targetmain
Add statictendril miscweb static site to projects listrepos/releng/gitlab-trusted-runner!30dzahnmainmain
Update README.mdrepos/sre/miscweb/statictendril!1dzahndzahn-main-patch-92112main
Customize query in GitLab

Related Objects
Search...

Event Timeline

Jelto created this task.Jun 23 2023, 9:06 AM
Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJun 23 2023, 9:06 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Jelto mentioned this in T300171: Move micro sites from Ganeti to Kubernetes and from Gerrit to GitLab.Jun 23 2023, 9:07 AM
CodeReviewBot added a comment.Jun 23 2023, 9:17 AM

jelto updated https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/30

Add statictendril miscweb static site to projects list

CodeReviewBot added a comment.Jun 23 2023, 9:19 AM

jelto merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/30

Add statictendril miscweb static site to projects list

Jelto triaged this task as Medium priority.Jun 23 2023, 9:20 AM

@Dzahn the image build for tendril/dbtree fails currently because there is no ./.pipeline/blubber.yaml file, see https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/jobs/110963.

So similar to bienvenida we need also a blubber.yaml file, see https://gitlab.wikimedia.org/repos/sre/miscweb/bienvenida/-/blob/master/.pipeline/blubber.yaml.

CodeReviewBot added a comment.Jun 23 2023, 9:24 AM

jelto updated https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/1

Update README.md

Jelto moved this task from Incoming to Work in Progress on the collaboration-services board.Jun 23 2023, 9:33 AM
gerritbot added a comment.Jun 23 2023, 9:49 AM

Change 930886 had a related patch set uploaded (by Jelto; author: Dzahn):

[operations/deployment-charts@master] miscweb: add statictendril release to miscweb staging

https://gerrit.wikimedia.org/r/930886

gerritbot added a project: Patch-For-Review.Jun 23 2023, 9:49 AM
Dzahn added a comment.Jun 23 2023, 3:04 PM
In T340182#8958058, @Jelto wrote:

So similar to bienvenida we need also a blubber.yaml file, see https://gitlab.wikimedia.org/repos/sre/miscweb/bienvenida/-/blob/master/.pipeline/blubber.yaml.

Thanks! Yea, I just forgot to create it. I just added it. https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/blob/main/.pipeline/blubber.yaml

CodeReviewBot added a comment.Jul 4 2023, 10:01 AM

jelto opened https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/2

fix image build for statictendril

CodeReviewBot added a comment.Jul 5 2023, 8:17 PM

dzahn closed https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/2

fix image build for statictendril

CodeReviewBot added a comment.Jul 5 2023, 8:17 PM

dzahn reopened https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/2

fix image build for statictendril

CodeReviewBot added a comment.Jul 5 2023, 8:18 PM

dzahn merged https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/2

fix image build for statictendril

Dzahn added a comment.Jul 5 2023, 8:45 PM

After this issue was fixed we now have a new issue:

This job could not start because it could not retrieve the needed artifacts: kokkuri:setup-variables

This is despite the kokkuri config being copied from existing working project:

https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/blob/main/.gitlab-ci.yml

vs

https://gitlab.wikimedia.org/repos/sre/miscweb/annualreport/-/blob/master/.gitlab-ci.yml

Also I don't even find the string "kokkuri" on wikitech.. are there docs?

CodeReviewBot added a comment.Jul 5 2023, 9:12 PM

dzahn merged https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/1

Update README.md

Jelto added a comment.Jul 6 2023, 9:47 AM

One comment on publishing images automatically on every merge (https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/commit/c27dc9d46b05550089fbe5b65d1f2748993cd788).

This is different from all other miscweb services. All other miscweb images need a dedicated tag pushed for every new image published to the registry.
The current configuration for tendril tags the image with the branch name main. This is not useful for deploying a new version to Kubernetes because we can not be sure if main is the new or the old version.

So we should either add

variables:
  PUBLISH_IMAGE_TAG: $KOKKURI_PIPELINE_TIMESTAMP-$BUILD_VARIANT

as a additional variable in .gitlab.ci.yml (see T338224) or go back to build on tags only (if: $CI_COMMIT_TAG && $CI_COMMIT_REF_PROTECTED).
Furthermore we should publish the images similar on all services, so either build on tags or on merges but then use the correct timestamp tag.

CodeReviewBot added a comment.Jul 6 2023, 12:07 PM

jelto opened https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/3

use timestamp as image tag

CodeReviewBot added a comment.Jul 6 2023, 6:40 PM

dzahn merged https://gitlab.wikimedia.org/repos/sre/miscweb/statictendril/-/merge_requests/3

use timestamp as image tag

gerritbot added a comment.Jul 10 2023, 5:19 PM

Change 930886 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: add statictendril release to miscweb staging

https://gerrit.wikimedia.org/r/930886

Dzahn changed the task status from Open to In Progress.Jul 10 2023, 5:28 PM
Dzahn raised the priority of this task from Medium to High.

deployed on k8s-staging

Dzahn added a comment.Jul 10 2023, 5:33 PM
 curl -I --resolve dbtree.wikimedia.org:30443:$(dig +short k8s-ingress-staging.discovery.wmnet) https://dbtree.wikimedia.org:30443 
HTTP/2 200 
date: Mon, 10 Jul 2023 17:32:17 GMT
server: istio-envoy
curl -I --resolve tendril.wikimedia.org:30443:$(dig +short k8s-ingress-staging.discovery.wmnet) https://tendril.wikimedia.org:30443 
HTTP/2 200 
date: Mon, 10 Jul 2023 17:32:50 GMT
server: istio-envoy
gerritbot added a comment.Jul 10 2023, 5:47 PM

Change 936775 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/deployment-charts@master] miscweb: add statictendril to eqiad and codfw k8s

https://gerrit.wikimedia.org/r/936775

gerritbot added a comment.Jul 10 2023, 5:55 PM

Change 936775 merged by jenkins-bot:

[operations/deployment-charts@master] miscweb: add statictendril to eqiad and codfw k8s

https://gerrit.wikimedia.org/r/936775

Dzahn added a comment.Jul 10 2023, 6:33 PM

deployed to eqiad and codfw:

curl -I --resolve tendril.wikimedia.org:30443:$(dig +short k8s-ingress-wikikube.svc.eqiad.wmnet) https://tendril.wikimedia.org:30443 
HTTP/2 200 
date: Mon, 10 Jul 2023 18:32:48 GMT
server: istio-envoy
curl -I --resolve tendril.wikimedia.org:30443:$(dig +short k8s-ingress-wikikube.svc.codfw.wmnet) https://tendril.wikimedia.org:30443 
HTTP/2 200 
date: Mon, 10 Jul 2023 18:32:42 GMT
server: istio-envoy
gerritbot added a comment.Jul 10 2023, 6:40 PM

Change 936785 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] trafficserver: switch dbtree/tendril to k8s backend

https://gerrit.wikimedia.org/r/936785

gerritbot added a comment.Jul 10 2023, 7:15 PM

Change 936785 merged by Dzahn:

[operations/puppet@production] trafficserver: switch dbtree/tendril to k8s backend

https://gerrit.wikimedia.org/r/936785

Dzahn updated the task description. (Show Details)Jul 10 2023, 7:27 PM
Dzahn added a comment.Jul 10 2023, 7:31 PM

switched traffic for dbtree and tendril.

19:24 < mutante> one easy way to confirm what serves it is the apache version
19:24 < mutante> for example: https://tendril.wikimedia.org/foo  show Apache 2.4.38
19:24 < mutante> but miscweb1003/2003 have  2.4.56 installed
Dzahn closed this task as Resolved.Jul 10 2023, 8:15 PM
Dzahn updated the task description. (Show Details)

20:14 < mutante> !log miscweb1003/miscweb2003 - rm -rf /srv/org/wikimedia/static-tendril

Dzahn mentioned this in T297605: Shutdown Tendril and dbtree.Jul 12 2023, 6:03 PM
gerritbot added a comment.Jul 12 2023, 6:04 PM

Change 937514 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] microsites: add monitoring for statictendril site for retired services

https://gerrit.wikimedia.org/r/937514

gerritbot added a comment.Jul 13 2023, 7:54 AM

Change 937514 merged by Jelto:

[operations/puppet@production] microsites: add monitoring for statictendril site for retired services

https://gerrit.wikimedia.org/r/937514

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL