Got a WIP PR here: https://github.com/wikimedia/ores/pull/345

Still need to slim down the production variant and handle the redis dep for the test variant.

A couple of questions here so far:

• Does the base image need to come from the wmf docker registry? If so, then it might make sense for us to create an optimized base image that has the scipy + enchant binaries ready to go.

• For the production image, we need to start a container to run the uwsgi service and also some separate containers to run the celery workers as well, can we specify that in blubberfile? Or do we need to create a deployment template for the helm chart somewhere?

In T210268#6167598, @ACraze wrote:

A couple of questions here so far:

• Does the base image need to come from the wmf docker registry?

Yes, and it must inherit from the SRE-provided base images.

• For the production image, we need to start a container to run the uwsgi service and also some separate containers to run the celery workers as well, can we specify that in blubberfile? Or do we need to create a deployment template for the helm chart somewhere?

Don't know that part; I don't know if we have any multi-container pods in production yet, sorry.

In T210268#6167607, @Jdforrester-WMF wrote:

In T210268#6167598, @ACraze wrote:

A couple of questions here so far:

• For the production image, we need to start a container to run the uwsgi service and also some separate containers to run the celery workers as well, can we specify that in blubberfile?

Adding @dduvall for this as he is the best person to answer this.

Or do we need to create a deployment template for the helm chart somewhere?

Not sure I understand this part of the question. A helm chart (including a Deployment template) will have to be created anyway, and architecturally speaking there should be different charts for the the celery worker and uwsgi. Does that help?

Don't know that part; I don't know if we have any multi-container pods in production yet, sorry.

We do, but not of the nature mentioned above. There are sidecar containers that help the main application providing functionality like TLS termination or metrics collection. But that's entirely different from the pattern of colocating in the same pod 2 containers that are actively serving requests (in one way or another). The pattern of multi-app-container pods however is best to be avoided as it will interfere with the ability to increase capacity for one of the 2 parts at will as well as allow the 2 parts to cause side effects to each other when under stress.

In T210268#6167598, @ACraze wrote:

• For the production image, we need to start a container to run the uwsgi service and also some separate containers to run the celery workers as well, can we specify that in blubberfile? Or do we need to create a deployment template for the helm chart somewhere?

We can build multiple images via PipelineLib using a .pipeline/config.yaml in the root of the repository. You can specify different variants of a single blubberfile be built or that different blubberfiles be used to build a variant. PipelineLib is documented on wikitech: https://wikitech.wikimedia.org/wiki/PipelineLib although you'll be a bit of a guinea pig for this use-case. I'm not aware of any other repos using it for this purpose; although it was designed with this purpose in mind :)

In T210268#6167598, @ACraze wrote:

• Does the base image need to come from the wmf docker registry? If so, then it might make sense for us to create an optimized base image that has the scipy + enchant binaries ready to go.

If these dependencies are currently installed via apt then blubber should be able to install them just fine. The python base images have the wikimedia apt repos added as sources.

@thcipriani thanks, PipelineLib seems to be what I was missing here :)