should we use blubber to generate these images?

Yes :) In fact, you have to.

should we use a single repo and have a new image generated per patch or a separate repo with a release process?

I'm not familiar with the app, but do you need 2 different images? Would one image with different app entrypoints suffice? If they are really different, but still use code from the same repo, I think it would be possible to publish mulitple blubber 'variants'.

I started working on this by 1) migrate the project to GitLab and 2) come up with pipeline to build and deploy the JARs. However, we currently do not have releng docker images suitable to be used as GitLab CI job-images.

• docker-registry.wikimedia.org/releng/java8:1.0.0-s1 uses java as entrypoint
• docker-registry.wikimedia.org/releng/maven-java8:1.0.0-s1 expects all kinds ZUUL parameters and expects to checkout and build whereas in GitLab land checkout is taken care of by GitLab itself.

In T326318#8743556, @pfischer wrote:

I started working on this by 1) migrate the project to GitLab and 2) come up with pipeline to build and deploy the JARs. However, we currently do not have releng docker images suitable to be used as GitLab CI job-images.

• docker-registry.wikimedia.org/releng/java8:1.0.0-s1 uses java as entrypoint

Isn't this something we can override? As far as I can see, that's what we do for our current zuul pipeline: https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/refs/heads/master/jjb/job-templates.yaml#160

• docker-registry.wikimedia.org/releng/maven-java8:1.0.0-s1 expects all kinds ZUUL parameters and expects to checkout and build whereas in GitLab land checkout is taken care of by GitLab itself.

Same thing, can we override? All the ZUUL parameters are probably only related to some weirdness of how zuul works, and can be ignored if we implement a different workflow on top of the same base image.

Take the comments above with a grain of salt, I don't know much at all about how gitlab CI works.

👋 howdy all! Release-Engineering-Team checking in :)

In T326318#8743556, @pfischer wrote:

I started working on this by 1) migrate the project to GitLab and 2) come up with pipeline to build and deploy the JARs. However, we currently do not have releng docker images suitable to be used as GitLab CI job-images.

• docker-registry.wikimedia.org/releng/java8:1.0.0-s1 uses java as entrypoint
• docker-registry.wikimedia.org/releng/maven-java8:1.0.0-s1 expects all kinds ZUUL parameters and expects to checkout and build whereas in GitLab land checkout is taken care of by GitLab itself.

Confusingly, most of the /releng images are specific for our Jenkins/docker CI system (hence all the ZUUL stuff in them).

You might find better base images in the operations/docker-images/production-images repo. I found a flink folder there (in case that's helpful)

Relatedly...I made a monster task: T332953: Migrate PipelineLib repos to GitLab where I made this offer

• Release-Engineering-Team will
  • Archive your gerrit repo
  • Make you a GitLab repo
  • Add needed folks to the repo
  • Translate your .pipeline/config.yaml to work with GitLab
• You will
  • Use GitLab and give us feedback (and we'll update our GitLab documentation/tooling as needed)
  • Be available to bug with questions if your tests break or repo ownership is confusing for RelEngers

tl;dr: I'm looking to pair up with teams who are migrating to GitLab. We can (I hope) help navigate this space.

Let me know if that's interesting/you have the time. You can drop into #wikimedia-releng on libera or #engineering-enablement on slack—or async here.

You might find better base images in the operations/docker-images/production-images repo. I found a flink folder there (in case that's helpful)

Ya this is the base image.

Please see the usage of kokkuri in mediawiki-event-enrichment .gitlab-ci.yaml, and the .pipeline/blubber.yaml config that builds a app docker image based on the flink one in production-images.

tl;dr

• blubber.yaml == Dockerfile. Use docker-registry.wikimedia.org/flink:x.x.x as the base to get Java and Flink dependencies, then use the blubber build steps to build and add your application dependencies into the image you will buld.

• Use the kokkuri jobs in .gitlab-ci to have gitlab build your image and publish it to docker-registry.wikimedia.org. @dduvall can probably help here ( I think you need your repo to be added as a trusted repo or something?)

pfischer opened https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/1

Draft: 👷 CI pipeline

pfischer opened https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/21

Added cirrus-streaming-updater

pfischer merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/1

👷 CI pipeline

pfischer opened https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/2

Draft: Pipeline

dancy merged https://gitlab.wikimedia.org/repos/releng/gitlab-trusted-runner/-/merge_requests/21

Added cirrus-streaming-updater

pfischer merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/2

Pipeline