We need a technical document describing how the Jenkins jobs are going to be isolated ( T47499 ). Most importantly the flow between Gerrit/Zuul/Jenkins/Nodepool/OpenStack and the servers/instances responsibilities.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | hashar | T60772 common gating job for mediawiki core and extensions | |||
| Resolved | hashar | T69216 Have unit tests of all wmf deployed extensions pass when installed together, in both PHP-Zend and HHVM (tracking) | |||
| Invalid | Ryasmeen | T90647 Create Jenkins builds for Editing across repositories (MobileFrontend, VisualEditor etc) | |||
| Declined | None | T50407 Jenkins: Setup Vagrant for some jobs (tracking) | |||
| Declined | None | T45266 Write and implement tests for Wikimedia's Apache configuration (redirects.conf, etc.) | |||
| Resolved | hashar | T47499 [EPIC] Run CI jobs in disposable VMs | |||
| Invalid | hashar | T86172 Write a migration plan for CI infra to the disposable VMs infrastructure | |||
| Resolved | hashar | T86171 Design the Jenkins isolation architecture | |||
| Declined | hashar | T97472 Create a Trusty labs image for CI isolation project |
Event Timeline
Comment Actions
I started writing the document at https://www.mediawiki.org/wiki/Continuous_integration/Architecture/Isolation . Will poke once ready.
Comment Actions
I have drawn a general overview of the envisioned architecture https://www.mediawiki.org/wiki/File:Integrationwikimediaci-architecture-isolation.svg
Comment Actions
I have updated the architecture document this week. A concern I had was to have all Jenkins runner to be able to access the Zuul mergers. During the transitions we will have slaves:
- gallium prod, public address
- lanthanum prod, public address
- labs instances
In the current system they all can access zuul.eqiad.wmnet which is the public IP of gallium. With the new systems all three kind of slaves would need to be able to reach the Zuul merger in labs subnet. We will have to open firewalls rules to permit traffic from the prod slaves (gallium and lanthanum) to the labs subnet IP of the Zuul mergers.
@zeljkofilipin has read the document and we had a quick 1/1 today about it. We scheduled three weekly one hours checkins to talk more about this project and other things.
Comment Actions
Nodepool has been deployed. The rest of the components are being slowly added as time allow. We have just started adding scandium as a Zuul merger in the labs infra (T95046).
I am getting the documentation updated on the wiki: https://www.mediawiki.org/wiki/Continuous_integration/Architecture/Isolation
Not much more design work will happen. The original inception back in beginning of 2015 is more or less what we have/are deploying.