Page MenuHomePhabricator

Mathoid CI Container Build
Closed, ResolvedPublic

Description

Build a container for mathoid that CI can promote. Figure out how to best utilize docker's image cache to do so. And figure out when and how to include the testing requirements in the container. As well as when and how to "promote" a tested container.

Related Objects

StatusSubtypeAssignedTask
ResolvedNone
InvalidNone
InvalidNone
ResolvedNone
Resolveddduvall
InvalidNone
DeclinedNone
Resolveddduvall
Resolveddduvall
Resolveddduvall
Resolveddduvall
Resolveddduvall
ResolvedNone
Resolveddduvall
Resolvedthcipriani
Resolveddduvall
Resolveddduvall
Resolvedthcipriani
Resolveddduvall
Resolvedthcipriani
ResolvedJoe
Resolveddduvall
Resolveddduvall
Resolveddduvall
Resolveddduvall
Resolvedakosiaris
Resolvedthcipriani
DeclinedNone
Resolvedthcipriani
Resolveddduvall
Resolveddduvall
InvalidNone
Resolveddduvall
Resolveddduvall
Resolveddduvall
InvalidNone
InvalidNone

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 7 2017, 6:20 PM

The Service runner had a patch ( https://github.com/wikimedia/service-runner/commit/f552b183ce9beaab7dff9417cad2c9c7d90f1872 ) to support generating the Dockerfile. Should be something like:

runner generate testing

Output would be a DockerFile.


For cache of requirements, I guess it is up to the developer to define the behavior they want. But I guess for a npm repo they would want something like:

COPY package.json ./
RUN npm install

Is that part of the image building step or is that already testing the software? :-]

For production, npm install --production should be used. Since npm install installs developments dependencies as well (which we don't want to have in the final image), this should be used for testing only.

thcipriani triaged this task as Medium priority.Feb 14 2017, 4:28 PM

So now that service-runner has the generate subcommand you can run something like:

$ cd services/mathoid
$ ../service-runner/service-runner.js generate --testing

And it will generate a testing Dockerfile for Mathoid in the $PWD:

services/mathoid/Dockerfile
FROM debian:jessie
RUN apt-get update && apt-get install -y nodejs nodejs-legacy npm git wget librsvg2-dev && rm -rf /var/lib/apt/lists/*
ENV NVM_DIR /usr/local/nvm
RUN wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.31.1/install.sh | bash && . $NVM_DIR/nvm.sh && nvm install 6.9.1
RUN mkdir /opt/service
ADD . /opt/service
WORKDIR /opt/service
RUN . $NVM_DIR/nvm.sh && nvm use 6.9.1 && npm install && npm dedupe
RUN groupadd -o -g 1000 -r rungroup && useradd -o -m -r -g rungroup -u 1000 runuser
USER runuser
ENV HOME=/home/runuser LINK=g++
ENV IN_DOCKER=1
CMD . $NVM_DIR/nvm.sh && nvm use 6.9.1 && npm test
hashar added a comment.EditedFeb 14 2017, 5:11 PM

The content from the DockerFile gets data from the repository package.json section "deploy":

"deploy": {
  "node": "6.9.1",
  "target": "debian",
  "dependencies": {
    "_all": [
      "librsvg2-dev"
    ]
  }
}

So one can easily use a specific NodeJS version (in this case 6.9.1 but could be system to use the distribution one).

Dependencies can be various system packages (under _all). Can have different version based on distro name (ubuntu, debian, an example is Graphoid).

For production we have the service::packages puppet define:

modules/mathoid/manifests/packages.pp
class mathoid::packages {
    service::packages { 'mathoid':
        pkgs     => ['librsvg2-2'],
        dev_pkgs => ['librsvg2-dev'],
    }
}
thcipriani renamed this task from Mathoid CI Container Image POC to Mathoid CI Container Build.Aug 11 2017, 6:02 PM
dduvall changed the status of subtask T173127: Establish blubber config in mathoid from Open to Stalled.Nov 14 2017, 11:45 PM
thcipriani changed the status of subtask T173127: Establish blubber config in mathoid from Stalled to Open.Dec 7 2017, 3:46 PM
thcipriani closed this task as Resolved.Aug 29 2018, 3:40 PM

Mathoid is currently running in production on Kubernetes. The container image being run by Kubernetes was built by the the service pipeline job that runs on Mathoid post-merge. Calling this task resolved.