Page MenuHomePhabricator

No test report files were found: job fails in jenkins but is shown as successful in raita
Closed, DeclinedPublic


Run fails in jenkins but is show as successful in raita, the later seems to be correct.
Probably because of this:

ERROR: Publisher 'Publish JUnit test result report' failed: No test report files were found. Configuration error?

Event Timeline

JanZerebecki raised the priority of this task from to Needs Triage.
JanZerebecki updated the task description. (Show Details)
JanZerebecki added a subscriber: JanZerebecki.
zeljkofilipin set Security to None.

When executing the cucumber command locally, the result directory is filled correctly. Last successful run was on Aug, 6. Failing runs have no build artifacts at all, so maybe this is not junit export related at all (Since dashBoard_*.xml should be present in any case and is not in the log/junit directory). @JanZerebecki added an ls to the job which asserted that log/junit is present but empty:

12:04:23 + ls -la /mnt/jenkins-workspace/workspace/browsertests-Wikidata-PerformanceTests-linux-firefox-sauce/log/junit
12:04:23 total 8
12:04:23 drwxr-xr-x 2 jenkins-deploy wikidev 4096 Oct 30 12:04 .
12:04:23 drwxr-xr-x 3 jenkins-deploy wikidev 4096 Oct 30 12:04 ..

From an IRC discussion on 2015-10-27 in Wikidata:

<Adrian_WMDE> hashar: Can you help us/me fixing
<Adrian_WMDE> It currently fails with `ERROR: Publisher 'Publish JUnit test result report' failed:
              No test report files were found. Configuration error?`

<hashar> seems some Junit xml file is not generated
<hashar> Adrian_WMDE: looks like no scenario matches the cucumber tags
<hashar> cucumber is invoked with --tags @performance_testing --tags @firefox
<hashar> and since no tests are run, no JUnit result is written and the Jenkins plugin that deal with JUnit files complains and fail the build

<Adrian_WMDE> But features/performance_test.feature matches that, and the output looks to me like it is recognized
<Adrian_WMDE> Also, I just ran `bundle exec cucumber --backtrace --color --verbose --format junit --out junit --tags @performance_testing --tags @firefox` locally and it ran

The job does CUCUMBER_TAGS=@performance_testing and cucumber is executed with:

bundle exec cucumber --backtrace --color --verbose \
   --format Cucumber::Formatter::Sauce --out log/junit \
   --tags @performance_testing --tags @firefox

It apparently only match a single feature:


  • features/performance_test.feature
# Wikidata item tests
# License:: GNU GPL v2+
# testing loading time of huge entities

@chrome @firefox @internet_explorer_10 @internet_explorer_11 @local_config @performance_testing
Feature: High performance

    Given Entity Italy defined in data/q38.json exists

  Scenario Outline: Loading a huge entity
    When I load the huge item <item>
    Then Header edit button should be there
      And Statement add button should be there

      | item          |
      | Italy         |

It runs just fine on my local machine. An oddity is whenever RAITA_URL is set, cucumber output is not shown. I suspect there is a background error related to Raita that is not being shown somehow :-(

I ran the job manually directly on the slave reusing the env variables and I apparently I can not reproduce :-(

I suppose you mean you cannot reproduce the issue, right? Fyi, it's indeed only one scenario that should be run, so that's alright.

I just used --expand in my cucumber call to get more/all the meaningful data. I don't suppose that would help here, but who knows. I'm running cucumber 2.0.0.

@zeljkofilipin is there any status update on this? To me it looks like the performance plugin for Jenkins is somewhat broken..

@Tobi_WMDE_SW sorry, I was traveling, still catching up. I will take a look probably next week.