Page MenuHomePhabricator

Missing XML files cause "Publish Performance test result report"
Closed, ResolvedPublic

Description

Summary

That is a regression in the Jenkins Performance plugin between version 2.0. and 2.2. I have rollback the plugin to 2.0 and the job pass just fine now.

Filled Upstream as JENKINS-41417 JENKINS-43438

Issue

https://integration.wikimedia.org/ci/job/selenium-GettingStarted/385/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/console and https://integration.wikimedia.org/ci/job/selenium-GettingStarted/386/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/console both failed due to the XML files being missing.

19:03:03 ERROR: Step ‘Publish Performance test result report’ aborted due to exception: 
19:03:03 java.io.FileNotFoundException: /srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/log/junit/*.xml (No such file or directory)
19:03:03 	at java.io.FileInputStream.open(Native Method)
19:03:03 	at java.io.FileInputStream.<init>(FileInputStream.java:146)
19:03:03 	at java.io.FileReader.<init>(FileReader.java:72)
19:03:03 	at hudson.plugins.performance.parsers.ParserDetector.detect(ParserDetector.java:21)
19:03:03 	at hudson.plugins.performance.parsers.ParserFactory.getParser(ParserFactory.java:26)
19:03:03 	at hudson.plugins.performance.PerformancePublisher.getParsers(PerformancePublisher.java:439)
19:03:03 	at hudson.plugins.performance.PerformancePublisher.perform(PerformancePublisher.java:481)
19:03:03 	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
19:03:03 	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
19:03:03 	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
19:03:03 	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
19:03:03 	at hudson.model.Build$BuildExecution.post2(Build.java:186)
19:03:03 	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
19:03:03 	at hudson.model.Run.execute(Run.java:1753)
19:03:03 	at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
19:03:03 	at hudson.model.ResourceController.execute(ResourceController.java:98)
19:03:03 	at hudson.model.Executor.run(Executor.java:405)

Below is 386:

1Started by upstream project "selenium-GettingStarted" build number 386
2originally caused by:
3 Started by user mattflaschen
4Building remotely on saucelabs-03 (BrowserTests) in workspace /srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests
5Using checkout strategy: Default
6Last Built Revision: Revision b1b5cd3014a2bd6074fe4981ecc6098d3bb5b25b (origin/master)
7 > git rev-parse --is-inside-work-tree # timeout=10
8Fetching changes from the remote Git repository
9 > git config remote.origin.url https://gerrit.wikimedia.org/r/mediawiki/extensions/GettingStarted # timeout=10
10Pruning obsolete local branches
11Fetching upstream changes from https://gerrit.wikimedia.org/r/mediawiki/extensions/GettingStarted
12 > git --version # timeout=10
13 > git fetch --tags --progress https://gerrit.wikimedia.org/r/mediawiki/extensions/GettingStarted +refs/heads/*:refs/remotes/origin/* --prune
14Checking out Revision b1b5cd3014a2bd6074fe4981ecc6098d3bb5b25b (origin/master)
15 > git config core.sparsecheckout # timeout=10
16 > git checkout -f b1b5cd3014a2bd6074fe4981ecc6098d3bb5b25b
17 > git rev-list b1b5cd3014a2bd6074fe4981ecc6098d3bb5b25b # timeout=10
18Cleaning workspace
19 > git rev-parse --verify HEAD # timeout=10
20Resetting working tree
21 > git reset --hard # timeout=10
22 > git clean -fdx # timeout=10
23 > git tag -a -f -m Jenkins Build #386 jenkins-BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests-386 # timeout=10
24No emails were triggered.
25[BrowserTests] $ /bin/bash -xe /tmp/hudson440161821117145517.sh
26+ mkdir -p /srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/log/junit
27[BrowserTests] $ /bin/bash -xe /tmp/hudson7183250079443586129.sh
28+ [[ beta == \b\e\t\a* ]]
29+ export MEDIAWIKI_API_URL=https://en.wikipedia.beta.wmflabs.org/w/api.php
30+ MEDIAWIKI_API_URL=https://en.wikipedia.beta.wmflabs.org/w/api.php
31++ /srv/deployment/integration/slave-scripts/bin/mw-api-siteinfo.py https://en.wikipedia.beta.wmflabs.org/w/api.php git_branch
32+ MEDIAWIKI_GIT_BRANCH=master
33+ git checkout -f origin/master
34HEAD is now at b1b5cd3... Add number of pages to dump_redis.php
35+ git reset --hard origin/master
36HEAD is now at b1b5cd3 Add number of pages to dump_redis.php
37+ git clean -x -q -d -f
38[BrowserTests] $ /bin/bash -xe /tmp/hudson3936795819665878917.sh
39++ ruby -e 'print RbConfig::CONFIG["ruby_version"]'
40+ RUBY_VERSION=2.1.0
41+ export GEM_HOME=/mnt/home/jenkins-deploy/.gem/2.1.0
42+ GEM_HOME=/mnt/home/jenkins-deploy/.gem/2.1.0
43+ bundle install --verbose
44Using rake 11.1.1
450: rake (11.1.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rake-11.1.1.gemspec
46Using ast 2.1.0
470: ast (2.1.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ast-2.1.0.gemspec
48Using parser 2.2.2.6
490: parser (2.2.2.6) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/parser-2.2.2.6.gemspec
50Using astrolabe 1.3.1
510: astrolabe (1.3.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/astrolabe-1.3.1.gemspec
52Using builder 3.2.3
530: builder (3.2.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/builder-3.2.3.gemspec
54Using ffi 1.9.17
550: ffi (1.9.17) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ffi-1.9.17.gemspec
56Using childprocess 0.6.2
570: childprocess (0.6.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/childprocess-0.6.2.gemspec
58Using diff-lcs 1.3
590: diff-lcs (1.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/diff-lcs-1.3.gemspec
60Using multi_json 1.12.1
610: multi_json (1.12.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multi_json-1.12.1.gemspec
62Using gherkin 2.12.2
630: gherkin (2.12.2) from /usr/share/rubygems-integration/2.1/specifications/gherkin-2.12.2.gemspec
64Using multi_test 0.1.2
650: multi_test (0.1.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multi_test-0.1.2.gemspec
66Using cucumber 1.3.20
670: cucumber (1.3.20) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/cucumber-1.3.20.gemspec
68Using i18n 0.8.1
690: i18n (0.8.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/i18n-0.8.1.gemspec
70Using faker 1.7.3
710: faker (1.7.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faker-1.7.3.gemspec
72Using yml_reader 0.7
730: yml_reader (0.7) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/yml_reader-0.7.gemspec
74Using data_magic 1.0
750: data_magic (1.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/data_magic-1.0.gemspec
76Using unf_ext 0.0.7.2
770: unf_ext (0.0.7.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/unf_ext-0.0.7.2.gemspec
78Using unf 0.1.4
790: unf (0.1.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/unf-0.1.4.gemspec
80Using domain_name 0.5.20170223
810: domain_name (0.5.20170223) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/domain_name-0.5.20170223.gemspec
82Using multipart-post 2.0.0
830: multipart-post (2.0.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/multipart-post-2.0.0.gemspec
84Using faraday 0.11.0
850: faraday (0.11.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday-0.11.0.gemspec
86Using http-cookie 1.0.3
870: http-cookie (1.0.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/http-cookie-1.0.3.gemspec
88Using faraday-cookie_jar 0.0.6
890: faraday-cookie_jar (0.0.6) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday-cookie_jar-0.0.6.gemspec
90Using faraday_middleware 0.11.0.1
910: faraday_middleware (0.11.0.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/faraday_middleware-0.11.0.1.gemspec
92Using headless 2.3.1
930: headless (2.3.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/headless-2.3.1.gemspec
94Using json 2.0.3
950: json (2.0.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/json-2.0.3.gemspec
96Using mediawiki_api 0.7.1
970: mediawiki_api (0.7.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mediawiki_api-0.7.1.gemspec
98Using net-http-persistent 2.9.4
990: net-http-persistent (2.9.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/net-http-persistent-2.9.4.gemspec
100Using page_navigation 0.10
1010: page_navigation (0.10) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/page_navigation-0.10.gemspec
102Using rubyzip 1.2.1
1030: rubyzip (1.2.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rubyzip-1.2.1.gemspec
104Using websocket 1.2.4
1050: websocket (1.2.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/websocket-1.2.4.gemspec
106Using selenium-webdriver 3.1.0
1070: selenium-webdriver (3.1.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/selenium-webdriver-3.1.0.gemspec
108Using watir 6.2.0
1090: watir (6.2.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/watir-6.2.0.gemspec
110Using page-object 2.0.0
1110: page-object (2.0.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/page-object-2.0.0.gemspec
112Using mime-types 2.99.3
1130: mime-types (2.99.3) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mime-types-2.99.3.gemspec
114Using netrc 0.11.0
1150: netrc (0.11.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/netrc-0.11.0.gemspec
116Using rest-client 1.8.0
1170: rest-client (1.8.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rest-client-1.8.0.gemspec
118Using rspec-core 2.99.2
1190: rspec-core (2.99.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rspec-core-2.99.2.gemspec
120Using rspec-expectations 2.99.2
1210: rspec-expectations (2.99.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rspec-expectations-2.99.2.gemspec
122Using syntax 1.2.1
1230: syntax (1.2.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/syntax-1.2.1.gemspec
124Using thor 0.19.4
1250: thor (0.19.4) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/thor-0.19.4.gemspec
126Using mediawiki_selenium 1.8.0
1270: mediawiki_selenium (1.8.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/mediawiki_selenium-1.8.0.gemspec
128Using powerpack 0.1.1
1290: powerpack (0.1.1) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/powerpack-0.1.1.gemspec
130Using rainbow 2.0.0
1310: rainbow (2.0.0) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rainbow-2.0.0.gemspec
132Using ruby-progressbar 1.7.5
1330: ruby-progressbar (1.7.5) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/ruby-progressbar-1.7.5.gemspec
134Using rubocop 0.34.2
1350: rubocop (0.34.2) from /mnt/home/jenkins-deploy/.gem/2.1.0/specifications/rubocop-0.34.2.gemspec
136Using bundler 1.7.4
1370: bundler (1.7.4) from /usr/share/rubygems-integration/all/specifications/bundler-1.7.4.gemspec
138Your bundle is complete!
139Use `bundle show [gemname]` to see where a bundled gem is installed.
140+ export SAUCE_ONDEMAND_USERNAME=wikimedia-jenkins
141+ SAUCE_ONDEMAND_USERNAME=wikimedia-jenkins
142+ [[ beta == \b\e\t\a* ]]
143+ echo 'MEDIAWIKI_PASSWORD=$selenium_user_beta'
144MEDIAWIKI_PASSWORD=$selenium_user_beta
145+ set +x
146+ bundle exec rake selenium
147[DEPRECATION] `last_comment` is deprecated. Please use `last_description` instead.
148/usr/bin/ruby2.1 -S bundle exec cucumber tests/browser --backtrace --verbose --color --format pretty --format Cucumber::Formatter::Sauce --out '/srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/log/junit' --tags ~@skip --tags @en.wikipedia.beta.wmflabs.org --tags @firefox
149Code:
150 * tests/browser/features/support/env.rb
151 * tests/browser/features/support/pages/editable_random_page.rb
152 * tests/browser/features/support/pages/registration_page.rb
153Failed to load 'md' programming language for file tests/browser/README.md: cannot load such file -- cucumber/md_support/md_language
154 * tests/browser/README.md [NOT SUPPORTED]
155Failed to load 'yml' programming language for file tests/browser/ci.yml: cannot load such file -- cucumber/yml_support/yml_language
156 * tests/browser/ci.yml [NOT SUPPORTED]
157 * tests/browser/environments.yml [NOT SUPPORTED]
158 * tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb
159
160Features:
161 * tests/browser/features/modal_on_editable_returnto.feature
162Parsing feature files took 0m0.009s
163
164@en.wikipedia.beta.wmflabs.org @firefox
165Feature: Modal on editable returnto page
166
167 Scenario: User sees modal call to action after registration # tests/browser/features/modal_on_editable_returnto.feature:4
168 Given I am at a random page # mediawiki_selenium-1.8.0/lib/mediawiki_selenium/step_definitions/navigation_steps.rb:1
169 When I register # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:1
170 Then I should be returned to the same article # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:8
171 And I should see a modal call to action # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:12
172 And one action is edit this page # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:16
173 And one action is edit a suggested page # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:20
174 And one action is no thanks, maybe later # tests/browser/features/step_definitions/modal_on_editable_returnto_steps.rb:24
175
1761 scenario (1 passed)
1777 steps (7 passed)
1780m46.571s
179Recording test results
180IRC notifier plugin: Sending notification to: #wikimedia-releng
181ERROR: Step ‘Publish Performance test result report’ aborted due to exception:
182java.io.FileNotFoundException: /srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/log/junit/*.xml (No such file or directory)
183 at java.io.FileInputStream.open(Native Method)
184 at java.io.FileInputStream.<init>(FileInputStream.java:146)
185 at java.io.FileReader.<init>(FileReader.java:72)
186 at hudson.plugins.performance.parsers.ParserDetector.detect(ParserDetector.java:21)
187 at hudson.plugins.performance.parsers.ParserFactory.getParser(ParserFactory.java:26)
188 at hudson.plugins.performance.PerformancePublisher.getParsers(PerformancePublisher.java:439)
189 at hudson.plugins.performance.PerformancePublisher.perform(PerformancePublisher.java:481)
190 at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:78)
191 at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
192 at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
193 at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
194 at hudson.model.Build$BuildExecution.post2(Build.java:186)
195 at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
196 at hudson.model.Run.execute(Run.java:1753)
197 at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
198 at hudson.model.ResourceController.execute(ResourceController.java:98)
199 at hudson.model.Executor.run(Executor.java:405)
200Archiving artifacts
201[PostBuildScript] - Execution post build scripts.
202[BrowserTests] $ /bin/bash -xe /tmp/hudson2462942571325681054.sh
203+ set +e
204+ set +x
205Removing potential png screenshots under /log/
206Done
207Email was triggered for: Failure - Any
208Sending email for trigger: Failure - Any
209Sending email to: mflaschen@wikimedia.org qa-alerts@lists.wikimedia.org samsmith@wikimedia.org
210Finished: FAILURE

Event Timeline

Mentioned in SAL (#wikimedia-releng) [2017-05-02T19:51:15Z] <hashar> Jenkins: rolling back Performance plugin from 2.2 to 2.0 due to an exception / failure to find a junit xml file. T164296

I thought maybe the Performance plugin would be to blame but apparently that is not the case. For a build, there is definitely a file /srv/jenkins-workspace/workspace/selenium-GettingStarted/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/log/junit/TEST-tests-browser-features-modal_on_editable_returnto.xml with the content:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="0" name="Modal on editable returnto page" skipped="0" tests="1" time="42.729813">
<testcase classname="Modal on editable returnto page" name="User sees modal call to action after registration" time="42.729813">
  <system-out/>
  <system-err/>
</testcase>
  <system-out>
    <![CDATA[]]>
  </system-out>
  <system-err>
    <![CDATA[]]>
  </system-err>
</testsuite>

Mentioned in SAL (#wikimedia-releng) [2017-05-02T19:59:36Z] <hashar> Regenerate jobs selenium-GettingStarted from JJB - T164296

That is a regression in the Jenkins Performance plugin between version 2.0. and 2.2. I have rollback the plugin to 2.0 and the job pass just fine now.

Filled Upstream as JENKINS-41417.

hashar triaged this task as Medium priority.May 2 2017, 8:17 PM
hashar updated the task description. (Show Details)
hashar moved this task from Backlog to Reported Upstream on the Upstream board.
hashar moved this task from Backlog to Reported upstream on the Jenkins board.
zeljkofilipin lowered the priority of this task from Medium to Low.May 29 2017, 10:30 AM
hashar claimed this task.

We nowadays use performance plugin 3.17 and upgraded at some point in the past.