The Container Cabal has agreed that consuming a bare Dockerfile from project repos would lead to indeterministically built and unmaintainable (from a security standpoint) container images, but there needs to be some simplified and specialized specification for a build manifest that will determine how the application container is eventually built. See meeting notes from 2017-02-14 and 2017-02-21.
Requirements (WIP)
- Base image from which the application container is derived (limited to those in a WMF hosted registry, maintained by ops).
- A project's system level dependencies (packages from limited WMF or verified third-party sources) perhaps categorized for specific purposes. For example:
- "core" or "all" as in required for any build of the image
- "build" as in only necessary as build-time app dependencies which can be removed before the final image is registered
- "test" as in dependencies only used by test suites
- Application level dependencies/libraries (npm, gem, pip, etc.)
- this should probably just delegate to the package manager's own specification so as not to cause duplication
- compiler should know how to turn this into respective Dockerfile instructions that would result in efficiently cached intermediate FS layers
- Application entrypoint
- Test suite entrypoints?
- defines low-level suites used to verify the built image that can either be run serially or in parallel
Open questions
- Does this specification need to cover anything k8s related or should it only be concerned with the building of images (not the running of them)? Unlike the Dockerfile specification, the k8s spec is well thought out and declarative.
Examples in the wild
- Services already has their own specification for service-runner to generate Dockerfiles. This might make a good starting point for a more generalized spec.
- A YAML formatted specification by Pearson that's part of their internally developed Jenkins plugin for a container based pipeline
- A container build-/run-time tool called dgr has its own specification. It seems rather to be quite complicated and still relies on ad-hoc scripts (via hook entrypoints) for constructing images, but there might be some aspects of it worth a look.