Page MenuHomePhabricator

operations-puppet-wmf-style-guide spurts warning trying to copy files to master
Closed, DeclinedPublic

Description

Found in /srv/jenkins/builds/operations-puppet-wmf-style-guide/*/workspace-files/*.tmp , the job https://integration.wikimedia.org/ci/job/operations-puppet-wmf-style-guide/ has some trouble copying files apparently:

Copying the source file 'modules/profile/manifests/phabricator/httpd.pp' from the workspace to the build folder 'b5926720.tmp' on the Jenkins master failed.
Seems that the path is relative, however an absolute path is required when copying the sources.
Is the file 'httpd.pp' contained more than once in your workspace?
Is the file 'modules/profile/manifests/phabricator/httpd.pp' a valid filename?
If you are building on a slave: please check if the file is accessible under '$JENKINS_HOME/[job-name]/modules/profile/manifests/phabricator/httpd.pp'
If you are building on the master: please check if the file is accessible under '$JENKINS_HOME/[job-name]/workspace/modules/profile/manifests/phabricator/httpd.pp'
java.io.IOException: Failed to copy modules/profile/manifests/phabricator/httpd.pp to /srv/jenkins/builds/operations-puppet-wmf-style-guide/9283/workspace-files/b5926720.tmp
	at hudson.FilePath.copyTo(FilePath.java:2191)
	at hudson.plugins.analysis.util.Files.copyFilesWithAnnotationsToBuildFolder(Files.java:80)
	at hudson.plugins.analysis.core.HealthAwareRecorder.copyFilesWithAnnotationsToBuildFolder(HealthAwareRecorder.java:351)
	at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:91)
	at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:298)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
	at hudson.model.Build$BuildExecution.post2(Build.java:186)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
	at hudson.model.Run.execute(Run.java:1881)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
Caused by: java.nio.file.NoSuchFileException: modules/profile/manifests/phabricator/httpd.pp
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
	at java.nio.file.Files.newInputStream(Files.java:152)
	at hudson.FilePath$CopyTo.invoke(FilePath.java:2247)
	at hudson.FilePath$CopyTo.invoke(FilePath.java:2239)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3069)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to integration-agent-docker-1010
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1788)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
		at hudson.remoting.Channel.call(Channel.java:998)
		at hudson.FilePath.act(FilePath.java:1069)
		at hudson.FilePath.act(FilePath.java:1058)
		at hudson.FilePath.copyTo(FilePath.java:2233)
		at hudson.FilePath.copyTo(FilePath.java:2188)
		at hudson.plugins.analysis.util.Files.copyFilesWithAnnotationsToBuildFolder(Files.java:80)
		at hudson.plugins.analysis.core.HealthAwareRecorder.copyFilesWithAnnotationsToBuildFolder(HealthAwareRecorder.java:351)
		at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:91)
		at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:298)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1881)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)

The job uses the git plugin to poll the repository and clones it to $WORKSPACE/gitpolling. That was done for polling the scm and displaying the list of changes since last build.

We then run the releng/operations-puppet container which already has a copy of puppet.git and fetches the patch independently. The checkstyle report refers to files there but they are not present on the host $WORKSPACE leading to the issue.

We should skip the fetching logic in releng/operations-puppet and instead use the sources that got fetched by the Jenkins plugin under $WORKSPACE/gitpolling.

Event Timeline

Mentioned in SAL (#wikimedia-operations) [2020-05-27T08:52:55Z] <hashar> contint1001: find /srv/jenkins/builds/operations-puppet-wmf-style-guide -type f -name '*.tmp' -delete # T253729

thcipriani subscribed.

@hashar is this still a problem?

I have removed the job in September 2020 with https://gerrit.wikimedia.org/r/c/integration/config/+/630211 since it:

  • the job got broken for several months as a result of this issue
  • used a deprecated Jenkins plugin T260565
  • there are lot of style complaints

My intent in 2017 was to provide SREs a report of the progress as I understood the ultimate goal was to fix all the styles issues and enforce them once done.