Page MenuHomePhabricator

Setup CI for operations/software/gerrit
Closed, ResolvedPublic

Description

Hi, we should setup CI for operations/software/gerrit/gerrit per @demon idea :)

We will need to install bazel which we could do in docker either by doing:

or

When building we should do cd <gerrit_dir> | bazel build release and then bazel build plugins/<plugin>

Details

Related Gerrit Patches:
integration/config : mastergerrit-docker: save cache in castor
integration/config : mastergerrit-docker: bind /src in the container
integration/config : masterGerrit: Add CI for operations/software/gerrit

Event Timeline

Paladox created this task.Mar 13 2018, 12:52 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 13 2018, 12:52 AM

@hashar hi, wondering if you could help us setup this image please for bazel?

This uses alot of ram 2+ gb so will need to be on a docker instance that has alot of ram available.

This requires nodejs and python so would need to extend the nodejs and python image :)

Or may @Legoktm knows? :)

The docker slaves we use for phan probably have enough memory for this. Probably a custom docker image makes sense...I guess the main question is how to install bazel.

We would need ops to import that to apt.wikimedia.org then.

demon moved this task from Bugs & stuff to Local hacks on the Gerrit board.Jun 9 2018, 2:28 AM
hashar renamed this task from Setup CI for operations/software/gerrit/gerrit to Setup CI for operations/software/gerrit.Jan 16 2019, 7:40 AM

Bazel provides Debian packages for Java 1.8. Guide: https://docs.bazel.build/versions/master/install-ubuntu.html#install-on-ubuntu

deploy/wmf/* branch has some python script that could use lint/tests

Paladox claimed this task.Mar 1 2019, 3:16 PM
Paladox triaged this task as Medium priority.
greg added a subscriber: greg.

@Paladox are you actually working on this?

Mentioned in SAL (#wikimedia-releng) [2019-07-08T13:38:26Z] <hashar> Build docker-registry.wikimedia.org/releng/bazel:0.1.0 # T189549

Change 493328 had a related patch set uploaded (by Hashar; owner: Paladox):
[integration/config@master] Gerrit: Add CI for operations/software/gerrit

https://gerrit.wikimedia.org/r/493328

@greg oh, really sorry, i missed your comment. Yup i am! @hashar seemed to address the comments (he merged the bazel image).

Indeed we now have a Docker image with Bazel 0.27.1 (docker-registry.wikimedia.org/releng/bazel:0.1.0), it lacks a bit of configuration for CI though, notably figure out how Bazel caching works and which bits / directory we want to save in the central cache. A few things I found yesterday is that Bazel allows to specify default command line arguments via bazelrc files. Eg something like:

/etc/bazelrc
build --repository_cache=/cache/repository_cache
build --disk_cache=/cache/disk_cache

Not that I know what are those caches for, would have to inspect them and check which parts could use to be cached.

For Gerrit, I have updated Paladox change ( https://gerrit.wikimedia.org/r/#/c/integration/config/+/493328/26 ), something I noticed is that Gerrit in wmf/stable-2.15 comes with a .bazelrc which hardcodes some cache directories to `~/.gerritcodereview:

.bazelrc
build --repository_cache=~/.gerritcodereview/bazel-cache/repository
build --disk_cache=~/.gerritcodereview/bazel-cache/cas

So I have to make sure those settings are always overriden in the entry point.

Change 493328 merged by jenkins-bot:
[integration/config@master] Gerrit: Add CI for operations/software/gerrit

https://gerrit.wikimedia.org/r/493328

Mentioned in SAL (#wikimedia-releng) [2019-07-10T13:30:20Z] <hashar> on zuul-merger: rm /srv/zuul/git/operations/software/gerrit due to T157818 # T189549

Change 521869 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] gerrit-docker: bind /src in the container

https://gerrit.wikimedia.org/r/521869

Change 521871 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] gerrit-docker: save cache in castor

https://gerrit.wikimedia.org/r/521871

hashar closed this task as Resolved.Jul 10 2019, 1:45 PM

Tested on https://gerrit.wikimedia.org/r/#/c/operations/software/gerrit/+/521868/ and it works. The build generate:

  • release.war
  • core.zip which contains the plugins .jar files

Change 521869 merged by jenkins-bot:
[integration/config@master] gerrit-docker: bind /src in the container

https://gerrit.wikimedia.org/r/521869

Change 521871 merged by jenkins-bot:
[integration/config@master] gerrit-docker: save cache in castor

https://gerrit.wikimedia.org/r/521871