Page MenuHomePhabricator

Headless VE - Docker image pipeline
Closed, ResolvedPublic

Description

Headless VE needs to use WMF supported docker image deployment pipeline.

Needed for T431497: Deploy Headless VE in k8s for technical pilot.

Done is

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Add editcheck-headless to projects.jsonrepos/releng/gitlab-trusted-runner!182kemayopatch-1main
Add GitLab CI pipeline to build and publish the docker imagerepos/mediawiki/services/editcheck-headless!1ottoT433514_editcheck_headless_dockermain
Customize query in GitLab

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
Openppelberg
Openppelberg
OpenNone
OpenEsanders
Opendchan
OpenGGoncalves-WMF
Openppelberg
Openppelberg
OpenNone
OpenNone
OpenMNeisler
OpenOttomata
Openppelberg
OpenNone
DeclinedNone
Openbmartinezcalvo
ResolvedBUG REPORTmedelius
OpenNone
Resolvedcjming
OpenNone
OpenNone
OpenEsanders
OpenNone
Opendchan
OpenNone
Opendchan
DuplicateNone
ResolvedOttomata

Event Timeline

For reference: chromium-render is an existing service that uses the deployment pipeline to build a nodejs + headless browser docker image.

Change #1319542 had a related patch set uploaded (by Ottomata; author: Ottomata):

[integration/config@master] Zuul, jjb: Add EditCheck Headless image pipeline jobs

https://gerrit.wikimedia.org/r/1319542

Change #1319543 had a related patch set uploaded (by Ottomata; author: Ottomata):

[mediawiki/extensions/VisualEditor@master] EditCheckHeadless: Add deployment pipeline config

https://gerrit.wikimedia.org/r/1319543

Patch up.

To test locally:

DOCKER_BUILDKIT=1 docker build -f ../../../.pipeline/editcheck-headless/blubber.yaml \
  --target production -t editcheck-headless:test .

docker run --rm -p 3000:3000 -p 50051:50051 editcheck-headless:test
curl -s 'http://localhost:3000/config?wiki=enwiki' | jq .
curl -s 'http://localhost:3000/check?wiki=enwiki&title=Kate%20Bush' | jq .

@Esanders @DLynch given @hashar's comments here, I think we should move editcheck-headless code outside of the extensions/VisualEditor repository. Perhaps a new gitlab project? Then we can use the newere GitLab CI supported pipeline to build the image.

repos/mediawiki/services/editcheck-headless seems reasonable, yes. The Editing team already maintains https://gitlab.wikimedia.org/repos/mediawiki/services/zotero and https://gitlab.wikimedia.org/repos/mediawiki/services/zotero-translators so at least have some familiarity with GitLab.

I've rewritten-and-imported all the editcheck-headless commits into the new repo. (And am cleaning up the files in VE.)

Change #1319542 abandoned by Ottomata:

[integration/config@master] Zuul, jjb: Add EditCheck Headless image pipeline jobs

Reason:

https://gitlab.wikimedia.org/repos/mediawiki/services/editcheck-headless/-/merge_requests/1

https://gerrit.wikimedia.org/r/1319542

Change #1319543 abandoned by Ottomata:

[mediawiki/extensions/VisualEditor@master] EditCheckHeadless: Add deployment pipeline config

Reason:

https://gitlab.wikimedia.org/repos/mediawiki/services/editcheck-headless/-/merge_requests/1

https://gerrit.wikimedia.org/r/1319543

Hi @brennen, does the new repo need to be allowed to use trusted runners to publish to docker-registry.wikimedia.org? The first merge to main branch tried but failed.

Ah, https://wikitech.wikimedia.org/wiki/GitLab/Gitlab_Runner/Trusted_Runners

FYI:

Please use the following Phabricator task template to create a access request: Task Template

The linked Task Template does not work.

I went ahead and made the merge request for it

@brennen, would it be possible to merge this in the next couple of days? We have a deadline for the headless VE project that is coming up. We're trying to get all the dependency work reviewed or done by the end of this week. Thank you so much!