Page MenuHomePhabricator

Grizzly: CI improvements
Open, MediumPublic

Description

Today Grizzly patch reviews are essentially a manual process involving V+2/C+2

Likewise, the deployment process is simple. Puppet updates a copy of the repo on the grafana server, and grr apply is run by the deployer to make the change live (in a nutshell puppet agent -t; grr apply).

Lots of room for improvements!

A few ideas to further explore:

  • Initial jenkins scaffolding/integration
  • Jsonnet lint/parse common entry points, e.g. slo_dashboards.jsonnet
  • Automatically execute grr testing commands, these inherently perform parsing too potentially making the above moot. (note: grr needs api access, but perhaps allowing jenkins a read-only key against our public grafana instance is workable)
    • grr snapshot (generates would-be dashboards a throwaway copies, useful for seeing exactly what will be deployed before deploying)
    • grr diff (similar to PCC, shows what would be changed vs current production state)
  • Gate/submit style deployment automation

Event Timeline

herron triaged this task as Medium priority.Mar 9 2023, 6:39 PM
herron created this task.

Change 896408 had a related patch set uploaded (by Herron; author: Herron):

[operations/grafana-grizzly@master] add tox json(net) linting and address issues raised

https://gerrit.wikimedia.org/r/896408

Change 896408 merged by Herron:

[operations/grafana-grizzly@master] add tox json(net) linting and address issues raised

https://gerrit.wikimedia.org/r/896408

Change 901138 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/grafana-grizzly@master] Move to demjson3 and install jsonnet-lint 0.19.1

https://gerrit.wikimedia.org/r/901138

Change 901139 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/grafana-grizzly@master] Add .gitreview

https://gerrit.wikimedia.org/r/901139

Change 901138 merged by JMeybohm:

[operations/grafana-grizzly@master] Move to demjson3 and install jsonnet-lint 0.19.1

https://gerrit.wikimedia.org/r/901138

Change 901139 merged by JMeybohm:

[operations/grafana-grizzly@master] Add .gitreview

https://gerrit.wikimedia.org/r/901139

Change 901598 had a related patch set uploaded (by Ayounsi; author: Ayounsi):

[integration/config@master] Zuul: [operations/grafana-grizzly] Add CI

https://gerrit.wikimedia.org/r/901598

Change 901598 abandoned by Hashar:

[integration/config@master] Zuul: [operations/grafana-grizzly] Add CI

Reason:

Abandoning for now, feel free to restore using Blubber/PipelineLib (see my last comment https://gerrit.wikimedia.org/r/c/integration/config/+/901598/comments/442006cd_d3583e6c ).

https://gerrit.wikimedia.org/r/901598

From my review comment at https://gerrit.wikimedia.org/r/c/integration/config/+/901598/comments/442006cd_d3583e6c

That should be done with Release Pipeline (Blubber) which has a documentation for Go at https://wikitech.wikimedia.org/wiki/PipelineLib/Guides/How_to_define_a_golang_test_pipeline

That should have everything you need and you can even add extra Debian packages to the base image (eg jsonlint or python3-demjson). I'd recommend to do the linting with the same implementation used by your application, so instead of jsonlint/python3-demjson, whatever is used by the go.

You can define an image with Blubber that would:

  • use docker-registry.wikimedia.org/golang:latest as a base image
  • run the command go install path/to/jsonnet-lint
  • copy some shell script holding your test commands:
  • define an entrypoint using that script

The image definition would be in the operations/rafana-grizzly repository in a file .pipeline/blubber.yaml . Something such as:

.pipeline/blubber.yaml
# syntax=docker-registry.wikimedia.org/repos/releng/blubber:v0.12.1
version: v4
variants:
  test:
    base: docker-registry.wikimedia.org/golang1.18
    apt:
      packages:
        - jsonlint
    builder:
      command:
        - go install path/to/jsonnet-lint
    copies: 
      - from: local
        source: ci.sh
        destination: /usr/local/bin/ci.sh
    entrypoint:
     - /usr/local/bin/ci.sh

The doc to build the image locally is at https://wikitech.wikimedia.org/wiki/Blubber/Download , that should be:

docker buildx build --target test -f blubber.yaml

Add a .pipeline/config.yaml that uses the "test" variant image:

.pipeline/config.yaml
- pipelines:
    test:
      blubberfile: blubber.yaml
    stages:
     - name: test

In the integration/config you would need to define the Jenkins jobs to build and run that test which is documented at https://wikitech.wikimedia.org/wiki/PipelineLib/Guides/How_to_configure_CI_for_your_project