search platform maven projects failing post merge build
Closed, ResolvedPublic

Description

Some of the maven project from the search platform team started failing their post-merge build with:

10:55:49 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:3.1.0:single (default) on project experimental-highlighter-elasticsearch-plugin: Failed to create assembly: Error adding file 'org.wikimedia.search.highlighter:experimental-highlighter-core:jar:6.4.2-SNAPSHOT' to archive: /src/experimental-highlighter-core/target/classes isn't a file. -> [Help 1]

This was observed after upgrading to discovery-parent-pom:1.20, but rolling back to 1.18 does not fix the issue, so probably unrelated.

The post merge build runs mvn clean install site site:stage.

The site and default lifecycles are not really meant to be run in the same execution. And running them separately seems to fix the issue, at least locally. So a better expression of that post-merge build should be: mvn clean install && mvn site site:stage.

Not entirely related, but during this investigation, I was wondering if the same version of maven was used on CI and locally. It seems to be the case, but only accidentally. Since all the search platform projects use maven-wrapper, which freezes the Maven version, it would be even better to not rely on the Maven version deployed on CI, but instead to run ./mvnw clean install && ./mvnw site site:stage.

Gehel created this task.Nov 1 2018, 1:22 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 1 2018, 1:22 PM
Gehel triaged this task as Normal priority.
Gehel added a subscriber: hashar.

That change is probably reasonably easy to implement in the JJB templates, but I'm lost there. I hope @hashar can provide some support.

TJones awarded a token.Nov 2 2018, 6:46 PM
TJones added a subscriber: TJones.

Thanks for hunting this down, @Gehel!

hashar added a comment.Nov 5 2018, 3:20 PM

A bit more context about the Docker container docker-registry.wikimedia.org/releng/java8.

Maven 3.5.2 is fetched from apache.org

The container sets:

ENV MAVEN_BIN=/opt/apache-maven-3.5.2/bin/mvn

It has a wrapper around mvn specially to enforce --batch-mode when needed and to rewrite some file path and pass it as an argument to --file. It runs $MAVEN_BIN.

I have a couple hacks in mind:

  • a quick one is to have Zuul to set a parameter: MAVEN_BIN=./mvnw
  • a probably better one is to have the mvn wrapper in the container to always use mvnw when it exists.

The job runs a single container which is being passed the goals: clean install site site:stage. We should be able to split the goals and reuse the states between two runs, though I would prefer to avoid that.

Gehel added a comment.Nov 5 2018, 3:36 PM

Why not drop the wrapper completely and just allow running an arbitrary command in the project directory? That way, we'll be able to configure ./mvnw clean install && ./mvnw site site:stage (or whatever we need). And not care about transient state needed to be propagated between containers. I'm not sure what value is that wrapper bringing, so I might be missing something.

Gehel claimed this task.Nov 6 2018, 6:23 PM

Change 472452 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Decouple maven install and site stages

https://gerrit.wikimedia.org/r/472452

Mentioned in SAL (#wikimedia-releng) [2018-11-08T14:30:49Z] <hashar> updated search-highlighter-maven-java8-docker-site-publish to decouple install and site goals | T208496

hashar added a comment.Nov 8 2018, 2:42 PM

We have updated the publish job for search/highlighter, I have manually rebuild the last build (which was for https://gerrit.wikimedia.org/r/#/c/470610/ ) and the outcome is a success:

https://integration.wikimedia.org/ci/job/search-highlighter-maven-java8-docker-site-publish/16/console
https://doc.wikimedia.org/search-highlighter/

If that works properly, I will refresh the other jobs, check them and then we can merge the ci config change :)

Gehel added a comment.Nov 8 2018, 5:17 PM

The build succeeded. The new docs are published at https://doc.wikimedia.org/search-highlighter/experimental/, the publication date has been updated, and the site looks fine (minus the usual broken links).

hashar closed this task as Resolved.Nov 9 2018, 3:09 PM

Looks good now :)

Change 472452 merged by jenkins-bot:
[integration/config@master] Decouple maven install and site stages

https://gerrit.wikimedia.org/r/472452