free some repositories from their unintended chain to mediawiki/core
Closed, ResolvedPublic

Description

Zuul is currently configured so that e.g. a run through the gate-and-submit pipeline for a patch in integration/config may wait on a patch in mediawiki/core. Or even retry, because the patch in core failed. There are other repositories that do not depend on any other repository, e.g. because they only use versioned dependencies.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 30 2015, 11:19 PM

Patch to integration/config that instead would duplicate and prefix jobs with the queue name: https://gerrit.wikimedia.org/r/#/c/243164/

Change 203290 had a related patch set uploaded (by Hashar):
Allow specifying jobs to use to merge queues in DependentPipelineManager

https://gerrit.wikimedia.org/r/203290

Change 243164 had a related patch set uploaded (by JanZerebecki):
Experiment YAML aliases and template inheritance

https://gerrit.wikimedia.org/r/243164

Change 243164 abandoned by Hashar:
Experiment YAML aliases and template inheritance

Reason:
No use for this right now. Can always revisit later if we end up thinking it is a good idea.

https://gerrit.wikimedia.org/r/243164

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 16 2015, 10:00 AM
hashar added a subscriber: hashar.

I have proposed upstream a patch to have the project templates to be based on the full name of the project instead of solely the base name. So we will be able to do something like:

project-templates:
  - name: test
    test:
      - {longname}-test
project:
  - name: mediawiki/core
    template:
      - test
  - name: pywikibot/core
    template:
      - test

That will cause Zuul to trigger respectively:

  • mediawiki-core-test
  • pywikibot-core-test

When currently we can only rely on the base name (in the example above core) which get both repositories to trigger core-test and thus to share the same queue in gate-and-submit.

Upstream patch is:

https://review.openstack.org/295237 Provide 'longname' to project templates

We have a bunch of Gerrit projects sharing the same base name:

$ rev projectnames.txt |cut -d\/  -f1|rev|sort|uniq -c|sort -rn|head
  15 deploy
   7 vendor
   5 wikipedia
   4 data
   4 dashboard
   3 wikimetrics
   3 plugins
   3 nginx
   3 core
   ...

Change 278733 had a related patch set uploaded (by Hashar):
Allow decoupling of npm-node-4.3

https://gerrit.wikimedia.org/r/278733

Change 278733 merged by jenkins-bot:
Allow decoupling of npm-node-4.3

https://gerrit.wikimedia.org/r/278733

Using YAML !!merge, we can extend the generic job and forge new ones with a prefix without duplicating code. I did so for wikimedia/portals ( T129591 ) with https://gerrit.wikimedia.org/r/278733

Change 281452 had a related patch set uploaded (by Hashar):
Allow decoupling of tox-jessie

https://gerrit.wikimedia.org/r/281452

Change 281452 merged by jenkins-bot:
Allow decoupling of tox-jessie

https://gerrit.wikimedia.org/r/281452

I have proposed upstream a patch to have the project templates to be based on the full name of the project instead of solely the base name. So we will be able to do something like:

project-templates:
  - name: test
    test:
      - {longname}-test
project:
  - name: mediawiki/core
    template:
      - test
  - name: pywikibot/core
    template:
      - test

That will cause Zuul to trigger respectively:

  • mediawiki-core-test
  • pywikibot-core-test

    When currently we can only rely on the base name (in the example above core) which get both repositories to trigger core-test and thus to share the same queue in gate-and-submit.

    Upstream patch is:

    https://review.openstack.org/295237 Provide 'longname' to project templates

The patch sent upstream as received two votes in favor. I am confident it will land soonish.

hashar moved this task from Backlog to Enhancements on the Zuul board.

Change 241660 had a related patch set uploaded (by Hashar):
Test mediawiki gate queue only has mw projects

https://gerrit.wikimedia.org/r/241660

Change 313028 had a related patch set uploaded (by Hashar):
Decouple repos from mediawiki gate queue

https://gerrit.wikimedia.org/r/313028

Change 320566 had a related patch set uploaded (by Hashar):
[VisualEditor] decouple from mediawiki gate-and-submit

https://gerrit.wikimedia.org/r/320566

Change 320566 merged by jenkins-bot:
[VisualEditor] decouple from mediawiki gate-and-submit

https://gerrit.wikimedia.org/r/320566

Change 203290 abandoned by Legoktm:
Allow specifying jobs to use to merge queues in DependentPipelineManager

https://gerrit.wikimedia.org/r/203290

Change 241660 merged by jenkins-bot:
Test mediawiki gate queue only has mw projects

https://gerrit.wikimedia.org/r/241660

Change 313028 merged by jenkins-bot:
[integration/config@master] Decouple repos from mediawiki gate queue

https://gerrit.wikimedia.org/r/313028

Mentioned in SAL (#wikimedia-releng) [2017-04-21T09:32:41Z] <hashar> Zuul: deploying "Decouple repos from mediawiki gate queue" 7a79f752363a / T107529

hashar closed this task as "Resolved".Fri, Apr 21, 9:33 AM
hashar claimed this task.

MediaWiki repositories are now in their own queue. That is enforced by tests in integration/config.