Page MenuHomePhabricator

Add support for a wmf-ci.yaml type file for wikimedia jenkins
Closed, DeclinedPublic

Description

Problem

Continuous Integration is hard to set up for repositories in the Wikimedia environment. Often this will require changes to the integration/config repo (e.g. editing a huge YAML file) and whitelisting of users (who may otherwise be unable to trigger some Jenkins jobs via Gerrit).

Proposed solution

Travis CI et similia only require a simple YAML file in the root directory of each code repository. Supporting such model can greatly streamline the addition of CI to repositories managed by small volunteer teams, decrease reliance on third-party services and ease the work of the few employees dedicated to CI infrastructure.


I discussed this on #wikimedia-releng and @mmodell thought it was a nice idea, he is the one who came up with the name for the file. But this will help us customize it per repo without needing to create a separate job.

<twentyafterfour> paladox: it would be cool to do something similar to travis, with a test config file, indeed.

What we will do is have a major improvement over .travis-ci which will allow us to define multiple tests at the same time instead of being limited to 1.

An example on how it will look

project:
  - name: hi
  - php:
       - 7
       - 5.6
       - 5.5
       - 5.4
       - 5.3
       - hhvm
script:
  - composer test

This is a mock and will need improvements to support extending our tests without making it complex.

Event Timeline

Would be nice to do a proof of concept based on https://github.com/travis-ci/travis-build If we get to partially support `.travis-ci file, it would be a great thing :-}

From a quick glance at it, it seems to compile the file to a plain shell. I am pretty sure the result would be tightly coupled with how their workers are defined, so it might not be so trivial to achieve compatibility. But that is worth a shot.

If we could support at list a subset of their commands, that would be a great first step.

Also theres https://github.com/travis-ci/travis-ci/blob/master/README.markdown which the other repos do different things. So we will need to also use those too. But I am not sure if we need to use them all.

mmodell moved this task from To Triage to Backlog on the Differential board.
hashar lowered the priority of this task from Medium to Low.Nov 3 2016, 9:11 PM

This proposal is selected for the Developer-Wishlist voting round and will be added to a MediaWiki page very soon. To the subscribers, or proposer of this task: please help modify the task description: add a brief summary (10-12 lines) of the problem that this proposal raises, topics discussed in the comments, and a proposed solution (if there is any yet). Remember to add a header with a title "Description," to your content. Please do so before February 5th, 12:00 pm UTC.

I dislike the idea because it is neither Jenkins nor Travis CI, but another solution that has no upstream. (I also don't understand this task's description; IIRC Travis CI allows to run tests in multiple environments at the same time?) So if we can set up a plain Travis CI instance, ideally by hiring some of their developers for the installation working with WMF engineers to do the puppetization, and hook that into Gerrit/Phabricator, that would be perfect; but I would like to avoid having another WMF-specific system that requires a WMF-specific syntax.

I dislike the idea because it is neither Jenkins nor Travis CI, but another solution that has no upstream. (I also don't understand this task's description; IIRC Travis CI allows to run tests in multiple environments at the same time?) So if we can set up a plain Travis CI instance, ideally by hiring some of their developers for the installation working with WMF engineers to do the puppetization, and hook that into Gerrit/Phabricator, that would be perfect; but I would like to avoid having another WMF-specific system that requires a WMF-specific syntax.

I don't think there's any requirement that it be proprietary to Wikimedia - the example in the description is using the travis format, and the links above suggest figuring out how much of travis-ci's code we can re-use.

[…]
I don't think there's any requirement that it be proprietary to Wikimedia - the example in the description is using the travis format, and the links above suggest figuring out how much of travis-ci's code we can re-use.

The file is proposed to be named wmf-ci.yaml; i. e., every repository to be developed both on GitHub and in WMF's environment or imported from one to the other would at the very least require a symbolic link to work. Additionally, if it is clear that the CI tests are always run at WMF, the barrier to deviate from "upstream" is very low, and people might be tempted to change php-7.0's meaning to php-7.0 + WMF-specific patches, producing potentially different results depending on if you use the WMF setup or Travis CI. (NB: I'm referring to Travis Ci here because the proposed format is very similar to it; there may be other solutions worth evaluating and using.)

This might be helpful:

https://github.com/vadimdemedes/trevor - Your own Travis CI to run tests locally.
"It reads .travis.yml and runs tests in all versions you requested, just like Travis CI. Now, you can test before push and keep your git history clean."

^ I think that might be too simplistic to actually be useful.