Page MenuHomePhabricator
Create Task
Maniphest T188686

Set up CI and github sync for new extra-analysis repo
Closed, ResolvedPublic
Actions

Description

@Gehel kindly set up a new repo for a new Elasticsearch plugin we're developing for CirrusSearch:

Could rel-eng please create the synchronization with github and enable CI for this repository?

Let me know if you need any other info.

Details

SubjectRepoBranchLines +/-
maven-site-publish-docker was not mounting /srcintegration/configmaster+2 -2
Add search/analysis-extra to jenkinsintegration/configmaster+9 -0
Customize query in gerrit

Event Timeline

TJones created this task.Mar 2 2018, 1:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 2 2018, 1:23 AM
Peachey88 added projects: Wikimedia-GitHub, Continuous-Integration-Config.Mar 2 2018, 1:29 AM
greg added a project: Diffusion-Repository-Administrators.Mar 5 2018, 4:59 PM
Reedy subscribed.Mar 5 2018, 5:02 PM

Empty https://github.com/wikimedia/search-extra-analysis created

TJones added a comment.Mar 5 2018, 7:37 PM

Thanks, @Reedy! Does anything else need to be done for the github synchronization?

Reedy added a comment.Mar 5 2018, 7:38 PM

IIRC, just some activity in the source repo, and then gerrit will try and replicate it all

TJones added a comment.Mar 5 2018, 7:45 PM

Cool. Hopefully I'll be committing soon. Thanks again!

TJones added a subscriber: dcausse.Mar 6 2018, 9:16 PM

@dcausse set up the CI here: https://gerrit.wikimedia.org/r/#/c/416743/

gerritbot subscribed.Mar 6 2018, 9:19 PM

Change 416743 had a related patch set uploaded (by Greg Grossmeier; owner: DCausse):
[integration/config@master] Add search/analysis-extra to jenkins

https://gerrit.wikimedia.org/r/416743

gerritbot added a project: Patch-For-Review.Mar 6 2018, 9:19 PM
gerritbot added a comment.Mar 7 2018, 11:02 AM

Change 416743 merged by jenkins-bot:
[integration/config@master] Add search/analysis-extra to jenkins

https://gerrit.wikimedia.org/r/416743

Stashbot subscribed.Mar 7 2018, 11:08 AM

Mentioned in SAL (#wikimedia-releng) [2018-03-07T11:08:43Z] <hashar> reloading Zuul for "Add search/analysis-extra to jenkins" | T188686

hashar closed this task as Resolved.Mar 7 2018, 11:06 PM
hashar assigned this task to dcausse.
hashar subscribed.

The CI job passed on the first change https://gerrit.wikimedia.org/r/#/c/415788/ . When it gets merged Gerrit will replicate the repository to GitHub.

TJones added a comment.Mar 8 2018, 7:20 PM

Gerrit has indeed now replicated the repo to GitHub. Thanks!

hashar reopened this task as Open.Mar 9 2018, 10:02 AM

Almost!

The job search-extra-analysis-maven-java8-docker-site-publish runs after a change has been merged and roughly do:

mvn clean install site site:stage
rsync src/target/staging <doc.wikimedia.org>

It fails because of a path issue. A slash is missing between src/target/staging and the project name extra-analysis:

00:00:55.757 [INFO] --- maven-site-plugin:3.7:stage (default-cli) @ extra-analysis ---
00:00:55.759 [INFO] Using this base directory for staging: /src/target/staging
00:00:55.765 [INFO] Pushing /src/target/site
00:00:55.765 [INFO]    >>> to file:///src/target/stagingextra-analysis
                                                       ^^^

And thus the rsync fails to find the directory src/target/staging:

rsync: change_dir "/srv/jenkins-workspace/workspace/search-extra-analysis-maven-java8-docker-site-publish//src/target/staging" failed: No such file or directory (2)
hashar added a comment.Mar 9 2018, 10:25 AM

Found it! The documentation is generated inside the container under /src but that directory is not mounted from the host. Hence when the container has finished the doc is gone and the publishing script finds nothing.

That is an error I made when creating the job. /src should be mounted from the host into the container.

gerritbot added a comment.Mar 9 2018, 10:25 AM

Change 417778 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] maven-site-publish-docker was not mounting /src

https://gerrit.wikimedia.org/r/417778

Stashbot added a comment.Mar 9 2018, 10:37 AM

Mentioned in SAL (#wikimedia-releng) [2018-03-09T10:37:47Z] <hashar> Update *-maven-java8-docker-site-publish jobs which were not mounting /src into the container | T188686

hashar added a comment.Mar 9 2018, 10:39 AM

Solved! And the site is at https://doc.wikimedia.org/search-extra-analysis/

The other similar repositories were affected as well, their doc site will be updated next time they get a change merged.

hashar closed this task as Resolved.Mar 9 2018, 10:39 AM
gerritbot added a comment.Mar 9 2018, 10:40 AM

Change 417778 merged by jenkins-bot:
[integration/config@master] maven-site-publish-docker was not mounting /src

https://gerrit.wikimedia.org/r/417778

TJones added a comment.Mar 9 2018, 2:59 PM

@hashar — so much happened while I was asleep! Thanks for getting everything into good shape!

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits