Page MenuHomePhabricator

Browser tests report as failures despite passing
Closed, DuplicatePublic

Description

The RelatedArticles build has failed for the last 2 attempts according to a visual scan of https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-RelatedArticles/ however if you dive into the test failures they actually passed.

Also note that the IRC bot thinks they've passed as well

<wmf-insecte> Yippee, build fixed!
9:26 AM Project selenium-RelatedArticles » chrome,beta-desktop,Linux,BrowserTests build #394: FIXED in 18 sec: > > https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-desktop,PLATFORM=Linux,label=BrowserTests/394/
9:26 AM Yippee, build fixed!
9:26 AM Project selenium-RelatedArticles » chrome,beta-mobile,Linux,BrowserTests build #394: FIXED in 38 sec: https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-mobile,PLATFORM=Linux,label=BrowserTests/394/
9:28 AM J

Looks like https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MobileFrontend/ may have the same problem.

Event Timeline

Jdlrobson created this task.May 2 2017, 4:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 2 2017, 4:32 PM
phuedx added a subscriber: phuedx.May 3 2017, 4:30 AM

No tests have failed. The build artifacts look correct. The "Publishing Performance test result report" step seems to be failing as it can't find the build artifacts, e.g. from https://integration.wikimedia.org/ci/job/selenium-RelatedArticles/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta-desktop,PLATFORM=Linux,label=BrowserTests/394/console:

16:26:08 ERROR: Step ‘Publish Performance test result report’ aborted due to exception: 
16:26:08 java.io.FileNotFoundException: /srv/jenkins-workspace/workspace/selenium-RelatedArticles/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta-desktop/PLATFORM/Linux/label/BrowserTests/log/junit/*.xml (No such file or directory)
16:26:08 	at java.io.FileInputStream.open(Native Method)
16:26:08 	at java.io.FileInputStream.<init>(FileInputStream.java:146)
16:26:08 	at java.io.FileReader.<init>(FileReader.java:72)
16:26:08 	at hudson.plugins.performance.parsers.ParserDetector.detect(ParserDetector.java:21)
16:26:08 	at hudson.plugins.performance.parsers.ParserFactory.getParser(ParserFactory.java:26)
16:26:08 	at hudson.plugins.performance.PerformancePublisher.getParsers(PerformancePublisher.java:439)
16:26:08 	at hudson.plugins.performance.PerformancePublisher.perform(PerformancePublisher.java:481)
16:26:08 	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
16:26:08 	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
16:26:08 	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
16:26:08 	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
16:26:08 	at hudson.model.Build$BuildExecution.post2(Build.java:186)
16:26:08 	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
16:26:08 	at hudson.model.Run.execute(Run.java:1753)
16:26:08 	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
16:26:08 	at hudson.model.ResourceController.execute(ResourceController.java:98)
16:26:08 	at hudson.model.Executor.run(Executor.java:405)

It looks like some variables – BROWSER, MEDIAWIKI_ENVIRONMENT, PLATFORM – aren't being replaced in that path.

phuedx added a subscriber: hashar.May 3 2017, 4:30 AM

@hashar: Could this be due to the recent Jenkins upgrade?

hashar added a comment.May 3 2017, 2:39 PM

Yes that is the same as T164296 indeed. Sorry again for the breakage. The root cause is a regression in the Jenkins Performance plugin.