The Release Engineering team has started a working group to discuss and consider our future continuous integration tooling. Please help!
The RelEng team is working with SRE to build a continuous delivery and deployment pipeline, as well as changing production to run things in containers under Kubernetes. We aim to improve the process of making changes to software behind our various sites by making it take less effort, happen faster, be less risky, and as automated as possible. The developers will have a better development experience, be more empowered, and more productive.
Wikimedia has had a CI system for many years now, but is based on versions of tools that are reaching the end of their useful life. Those tools need to be upgraded, and this will probably require further changes due to how the new versions function. This is a good point to consider what tools and functionality we need and want.
The working group is tasked to consider the needs and wants, and evaluate the available options, and make a recommendation of what to use in the future. The deadline is March 25. The work is being documented at https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/CI_Futures_WG and we're currently collecting requirements and candidates to evaluate.
We would welcome any feedback on those! Via IRC (#wikimedia-pipeline), on the talk page of the working group's wiki page above, or as a comment to this blog post.
Event Timeline
I like the approach Github has gone with the beta of Github actions. What I like about it, is how super easy it is to integrate whatever tool you need in the pipeline, as long as you have it as a Docker container and you set a couple of labels. I tried it with a couple of projects and it was amazing how easy it was to add new tools. However I understand there are other considerations too :)
I second @Peter 's comment!
One thing that frustrates me is if I want to do anything with CI right now it requires writing a patch and getting it reviewed. Given a limited amount of people can review code, and I don't understand the CI code and how to test it, this creates an unnecessary tax on these kinds of tasks and usually dissuade me from using CI as much as I would.
To give a concrete example, recently, we wanted to host a storybook instance on wmflabs, and while in theory it is supported by our CI pipeline, it involved code writing and code review. In contrast, for personal projects I use netlify.com which automatically deploys from github on every pull request and then publishes all the contents of a folder of static files onto its server. This requires minimal setup cost (if any) - I press a few buttons and fill in some text telling it what command to run and it works! I'd love to see our own CI pipeline becoming just as hassle-free. A sign of success with the new pipeline could be measured in terms of number of tasks tagged with Release-Engineering-Team
Two particular and concrete problems with the current process that frustrate me are:
- being able to run a command within the deploy/merge process (I have an RFC open for this: T199004)
- being able to deploy a folder of static files to a URL without having to write code (basically what heroku and netlify.com support but specifically to a wmflabs instance)
I think the way npm test runs in our CI pipeline is a great example of what happens when we provide flexible ways for developers to do things. While originally intended just to lint code, this is now being excised for a wide variety of things - code coverage, checking SVG assets are minified to name a few. I'd love to see similar flexible techniques for doing things at times other than code review.