Set up CI and github sync for new extra-analysis repo
Closed, ResolvedPublic

Description

@Gehel kindly set up a new repo for a new Elasticsearch plugin we're developing for CirrusSearch:

Could rel-eng please create the synchronization with github and enable CI for this repository?

Let me know if you need any other info.

TJones created this task.Mar 2 2018, 1:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 2 2018, 1:23 AM
TJones added a comment.Mar 5 2018, 7:37 PM

Thanks, @Reedy! Does anything else need to be done for the github synchronization?

Reedy added a comment.Mar 5 2018, 7:38 PM

IIRC, just some activity in the source repo, and then gerrit will try and replicate it all

TJones added a comment.Mar 5 2018, 7:45 PM

Cool. Hopefully I'll be committing soon. Thanks again!

Change 416743 had a related patch set uploaded (by Greg Grossmeier; owner: DCausse):
[integration/config@master] Add search/analysis-extra to jenkins

https://gerrit.wikimedia.org/r/416743

Change 416743 merged by jenkins-bot:
[integration/config@master] Add search/analysis-extra to jenkins

https://gerrit.wikimedia.org/r/416743

Mentioned in SAL (#wikimedia-releng) [2018-03-07T11:08:43Z] <hashar> reloading Zuul for "Add search/analysis-extra to jenkins" | T188686

hashar assigned this task to dcausse.Mar 7 2018, 11:06 PM
hashar closed this task as Resolved.
hashar added a subscriber: hashar.

The CI job passed on the first change https://gerrit.wikimedia.org/r/#/c/415788/ . When it gets merged Gerrit will replicate the repository to GitHub.

TJones added a comment.Mar 8 2018, 7:20 PM

Gerrit has indeed now replicated the repo to GitHub. Thanks!

hashar reopened this task as Open.Mar 9 2018, 10:02 AM

Almost!

The job search-extra-analysis-maven-java8-docker-site-publish runs after a change has been merged and roughly do:

mvn clean install site site:stage
rsync src/target/staging <doc.wikimedia.org>

It fails because of a path issue. A slash is missing between src/target/staging and the project name extra-analysis:

00:00:55.757 [INFO] --- maven-site-plugin:3.7:stage (default-cli) @ extra-analysis ---
00:00:55.759 [INFO] Using this base directory for staging: /src/target/staging
00:00:55.765 [INFO] Pushing /src/target/site
00:00:55.765 [INFO]    >>> to file:///src/target/stagingextra-analysis
                                                       ^^^

And thus the rsync fails to find the directory src/target/staging:

rsync: change_dir "/srv/jenkins-workspace/workspace/search-extra-analysis-maven-java8-docker-site-publish//src/target/staging" failed: No such file or directory (2)

Found it! The documentation is generated inside the container under /src but that directory is not mounted from the host. Hence when the container has finished the doc is gone and the publishing script finds nothing.

That is an error I made when creating the job. /src should be mounted from the host into the container.

Change 417778 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] maven-site-publish-docker was not mounting /src

https://gerrit.wikimedia.org/r/417778

Mentioned in SAL (#wikimedia-releng) [2018-03-09T10:37:47Z] <hashar> Update *-maven-java8-docker-site-publish jobs which were not mounting /src into the container | T188686

Solved! And the site is at https://doc.wikimedia.org/search-extra-analysis/

The other similar repositories were affected as well, their doc site will be updated next time they get a change merged.

hashar closed this task as Resolved.Mar 9 2018, 10:39 AM

Change 417778 merged by jenkins-bot:
[integration/config@master] maven-site-publish-docker was not mounting /src

https://gerrit.wikimedia.org/r/417778

TJones added a comment.Mar 9 2018, 2:59 PM

@hashar — so much happened while I was asleep! Thanks for getting everything into good shape!