- Title of session: Using kokkuri to build container images on GitLab CI
- Session description:
In this session we will present the tool kokkuri. kokkuri standardizes GitLab CI pipelines for container image builds by providing a CI-template which can be included in other projects. kokkuri can use existing blubber configuration to build container images. Blubber is Wikimedia's declarative abstraction for secure, yaml-based definitions for service images. Any project which uses container images and blubber can benefit from this standardization and build production-ready container images with a few lines of yaml code.
We will cover the basics of blubber, the usage of kokkuri, different use cases and some example projects.
- Username for contact: @Jelto @brennen
- Session duration (25 or 50 min): 25
- Session type (presentation, workshop, discussion, etc.): presentation
- Language of session (English, Arabic, etc.): English
*bold text Prerequisites (some Python, etc.): containers, a bit of blubber
- Any other details to share?:
- Interested? Add your username below:
Notes from session:
Using kokkuri to build container images on GitLab CI
Date and time: May 3, 14:30
Relevant links
- Phabricator task: https://phabricator.wikimedia.org/T361937
- Session slides: https://docs.google.com/presentation/d/1JpqRz7EnPLuOQjwEsiiwv_hs210dTg-se0mItlXl9pg/
- Kokkuri: https://gitlab.wikimedia.org/repos/releng/kokkuri/
- Blubber: https://gitlab.wikimedia.org/repos/releng/blubber
Presenter
@[[phab:p/Jelto/|Jelto]] & @[[phab:p/brennen/|brennen]]
Participants
Notes
- Jelto: SRE at the Foundation, working much of time on GitLab.
- Agenda:
basics of containers
image building with blubber
CI image building with Kokkuri
Putting it all together
Examples
Q&A
Presentation:
Basic overview of containers
what is a container, what is an image, etc
dockerfile -> docker build -> image -> docker run -> Container
Blubber
Writing Dockerfiles is tricky, lots of ways to do it, not ideal for applications you want to deploy at scale, security concerns, etc.
Blubber mostly replaces writing a detailed Dockerfile
Declarative, structured YAML
Mostly supports Docker features, adds some - variants, etc.
Standardizes images to be created
image building with blubber
blubber.yaml -> docker build -> Image -> docker build -> container -> (dockerd) -> (buildkitd)
Demo of a basic .pipeline/blubber.yaml
kokkuri
replicates something like pipelineLib as is to Jenkins
reusable GitLab CI templates
building, testing, publishing of images
gitlab.wikimedia.ord/repos/releng/kokkuri
Include a predefined set of jobs from the kokkuri repo, you should get image building with relatively little effort
Gitlab
WMF has self-hosted at WMF
free to use for wiki projects including CI
login with wikitech account
gitlab.wikimedia.org
approval required - reach out to brennen, Andre, or Jelto
blubber.yaml <- gitlab-ci.yml -> kokkuri:build-image -> image
uses cloud runners, shared runners, trusted runners to go to the registry
examples
annual review report
mathoid: https://gitlab.wikimedia.org/repos/mediawiki/services/mathoid/
Sample pipelines: https://gitlab.wikimedia.org/repos/mediawiki/services/mathoid/-/pipelines
repos/releng/gitlab-trusted-runner lists projects allowed access to trusted runners (for publishing to production registry)
Questions
where does the image live in the kokkuri example?
- the image would be built on the runner
- it is not pushed to a shared registry
- build the image in your software and then it uses kokkuri for running tests and pushing to a registry.
Is there a tutorial? how do I use it in toolforge? do I need to move to gitlab, create pipelines? but is that all the steps?
- in MW, there is a guide (entry documentation) that should help out with using it for the first time: https://www.mediawiki.org/wiki/GitLab/Workflows/Deploying_services_to_production