Page MenuHomePhabricator

Homer: setup CI for static data repositories
Closed, ResolvedPublic

Description

Setup some CI validation for the static data repositories of Homer to ensure data consistency and avoid human errors.

In particular:

  • YAML file validation
  • Jinja2 templates validation

Event Timeline

Volans triaged this task as Medium priority.Feb 10 2020, 7:53 AM
Volans created this task.

Change 571231 had a related patch set uploaded (by Volans; owner: Volans):
[operations/homer/public@master] config: lint YAML files

https://gerrit.wikimedia.org/r/571231

Change 571232 had a related patch set uploaded (by Volans; owner: Volans):
[operations/homer/public@master] config: add tox env to lint YAML files

https://gerrit.wikimedia.org/r/571232

Change 571233 had a related patch set uploaded (by Volans; owner: Volans):
[operations/homer/public@master] templates: add unit tests

https://gerrit.wikimedia.org/r/571233

Change 571236 had a related patch set uploaded (by Volans; owner: Volans):
[integration/config@master] Add tox job for operations/homer/public

https://gerrit.wikimedia.org/r/571236

Change 571236 merged by jenkins-bot:
[integration/config@master] Add tox job for operations/homer/public

https://gerrit.wikimedia.org/r/571236

Change 571231 merged by Volans:
[operations/homer/public@master] config: lint YAML files

https://gerrit.wikimedia.org/r/571231

Change 571232 merged by Volans:
[operations/homer/public@master] config: add tox env to lint YAML files

https://gerrit.wikimedia.org/r/571232

Change 571233 merged by Volans:
[operations/homer/public@master] templates: add unit tests

https://gerrit.wikimedia.org/r/571233

Change 571367 had a related patch set uploaded (by Volans; owner: Volans):
[operations/homer/mock-private@master] config: add tox env to lint YAML files

https://gerrit.wikimedia.org/r/571367

Change 571369 had a related patch set uploaded (by Volans; owner: Volans):
[integration/config@master] Add tox job for operations/homer/mock-private

https://gerrit.wikimedia.org/r/571369

Change 571369 merged by jenkins-bot:
[integration/config@master] Add tox job for operations/homer/mock-private

https://gerrit.wikimedia.org/r/571369

Mentioned in SAL (#wikimedia-releng) [2020-02-10T22:24:36Z] <James_F> Zuul: Add tox for operations/homer/mock-private T244690

Looked briefly at this today with https://gerrit.wikimedia.org/r/#/c/operations/homer/mock-private/+/571374/

Gerrit stream event *is* happening:

{
  "author": {
    "name": "Thcipriani",
    "email": "tcipriani@wikimedia.org",
    "username": "thcipriani"
  },
  "approvals": [
    {
      "type": "Verified",
      "description": "Verified",
      "value": "0"
    },
    {
      "type": "Code-Review",
      "description": "Code-Review",
      "value": "0"
    }
  ],
  "comment": "Patch Set 1:\n\nrecheck",
  "patchSet": {
    "number": 1,
    "revision": "266c3a7fbe9cd13d4b20c06031cf36710d1654f4",
    "parents": [
      "0a359bb6fbb2b8729f85ee1de98404531b73e397"
    ],
    "ref": "refs/changes/74/571374/1",
    "uploader": {
      "name": "Legoktm",
      "email": "legoktm@xxxxx",
      "username": "legoktm"
    },
    "createdOn": 1581374528,
    "author": {
      "name": "Legoktm",
      "email": "legoktm@xxxxx",
      "username": "legoktm"
    },
    "kind": "REWORK",
    "sizeInsertions": 2,
    "sizeDeletions": 0
  },
  "change": {
    "project": "operations/homer/mock-private",
    "branch": "master",
    "id": "Id3ca22b1c380f5019fbac4f74a35514d94b919aa",
    "number": 571374,
    "subject": "[DNM] testing jenkins",
    "owner": {
      "name": "Legoktm",
      "email": "legoktm@xxxxx",
      "username": "legoktm"
    },
    "url": "https://gerrit.wikimedia.org/r/571374",
    "commitMessage": "[DNM] testing jenkins\n\nChange-Id: Id3ca22b1c380f5019fbac4f74a35514d94b919aa\n",
    "createdOn": 1581374528,
    "status": "NEW"
  },
  "project": "operations/homer/mock-private",
  "refName": "refs/heads/master",
  "changeKey": {
    "id": "Id3ca22b1c380f5019fbac4f74a35514d94b919aa"
  },
  "type": "comment-added",
  "eventCreatedOn": 1581541895
}

That should trigger the test pipeline: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/master/zuul/layout.yaml#395

But there's nothing in the /var/log/zuul/debug.log on contint1001 referencing *this* patchset

[thcipriani@contint1001 ~]$ grep '571374' /var/log/zuul/debug.log
[thcipriani@contint1001 ~]$

I can see that zuul *is* listening for events from that event stream, though:

$ ssh -p 29418 gerrit.wikimedia.org -- gerrit show-connections -w
Session    User            Remote Host
--------------------------------------------------------------
2eaf1dda   jenkins-bot     contint1001.wikimedia.org
49fe61eb   jenkins-bot     contint1001.wikimedia.org

This seems...bizarre.

Mentioned in SAL (#wikimedia-releng) [2020-02-12T21:36:35Z] <thcipriani> reloading zuul in an attempt to troubleshoot T244690

We worked it out; this repo is inheriting its ACL from labs/private, which Blocks most users, including JenkinsBot.

Change 571367 merged by jenkins-bot:
[operations/homer/mock-private@master] config: add tox env to lint YAML files

https://gerrit.wikimedia.org/r/571367