In the last week, the wikidata sauce browser test always got aborted due to reaching the 4 hour time out:
Description
Details
Project | Branch | Lines +/- | Subject | |
---|---|---|---|---|
integration/config | master | +6 -8 | Disable Raita and enable Cucumber pretty formatter for browsertests-Wikidata* jobs. |
Related Objects
Event Timeline
Namely:
Build | Start | Duration |
---|---|---|
#523 | Feb 18th 01:22 UTC | 2 hours 44 min |
#524 | Feb 19th 01:22 UTC | 1 day 19 hours (stuck by IRC plugin) |
#525 | Feb 20th 20:56 UTC | 4 hours (killed by timeout) |
Build #524 reference a single git change compared to #523: https://github.com/wmde/WikidataBrowserTests/commit/3bf7ebf00b0803942d4c5f573e5be8a87683180c
Maybe that causes the tests to take too long.
I manually hacked the job configuration to checkout other commits. The tests definitely run (used strace on cucumber to confirm it was hitting sauce labs).
A super annoyance is that cucumber emits NO output what soever until it is complete.
Change 275435 had a related patch set uploaded (by Zfilipin):
WIP Fix various problems with browsertests-Wikidata* jobs
The job https://integration.wikimedia.org/ci/view/BrowserTests/view/Wikidata/job/browsertests-Wikidata-WikidataTests-linux-firefox/127/consoleFull got stuck. I went on the machine and did a strace -s 11024 -ewrite,read -f -p <PID OF cucumber> and the test suite repeatedly spam:
{ "args" : [], "script" : "return jQuery.active" }
Which indicates that something repeatedly poll the browser until jQuery.active but iterate infinitely when it should dies out after X seconds / iterations.
The scenario was Scenario: Select a property:
Scenario: Select a property # features/reference.feature:71 When I click the statement edit button # features/step_definitions/statement_steps.rb:23 And I click the reference add button # features/step_definitions/reference_steps.rb:9 And I select the snak property stringprop # features/step_definitions/statement_steps.rb:46 <Build was aborted by hashar>
de66fffab3964f1cb7c9af607dc543fa is a Sauce job from browsertests-Wikidata-WikidataTests-linux-chrome-sauce/325 Jenkins job.
Jenkins error message is ERROR Job is not in progress Selenium::WebDriver::Error::WebDriverError) and Sauce error message is: Test exceeded maximum duration after 1800 seconds. That is 30 minutes. For just one scenario.
Looks like the job gets stuck while executing script: "return jQuery.active". It is called from [[ https://github.com/wmde/WikidataBrowserTests/blob/master/tests/browser/features/support/modules/entity_module.rb#L115 | EntityPage#ajax_wait ]].
The problem is not reproducible while using local browser, but it is reproducible while using Sauce.
Example code on how to do timeouts, used in [[ https://phabricator.wikimedia.org/diffusion/EVED/browse/master/modules/ve-mw/tests/browser/features/support/pages/visual_editor_page.rb;c930ac2779a1877ab2b5fa93ff9a56a2676f652e$203 | VisualEditorPage#visual_editor_element ]], for example:
def ajax_wait Timeout.timeout(5) do sleep(1.0 / 3) while execute_script('return jQuery.active') != 0 end sleep 1 true end
Change 275435 merged by jenkins-bot:
Disable Raita and enable Cucumber pretty formatter for browsertests-Wikidata* jobs.