Page MenuHomePhabricator

Make browsertests for CirrusSearch run on every submitted patch with proper CI infrastructure rather than a bot
Closed, DeclinedPublic

Description

We have recently done a bit of refactoring to the browser tests for cirrussearch to get the time they take to run under control. Using parallel_cucumber with 4 processes they are down to around 5 minutes (or more depending on load of the server tests are running against).

This should initially be non-voting, but will be made to vote once we see that it doesn't have intermittent failures and such.

Event Timeline

EBernhardson raised the priority of this task from to Needs Triage.
EBernhardson updated the task description. (Show Details)
EBernhardson added a subscriber: EBernhardson.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 6 2015, 5:23 PM

I should note that while they are called browsertests since they are in cucumber, the majority of the tests now talk directly to the api and do not use the browser. If it makes things easier we can run only the @api tag and skip saucelabs,

Running just the api tests (bundle exec parallel_cucumber features/ -o '--tags @api) in four processes against a 4 core labs instance does 394 scenarios in ~4 minutes.
Running the entire suite (bundle exec parallel_cucumber features/) in four processes against a 4 core labs instance does 502 scenarios in ~5.5 minutes

@dduvall added support for that to the CI infrastructure (T103039).

Reading has an epic about it T104315: [GOAL]: Set up testing environments that run browser tests on all patches. and they now have browser tests voting for some repositories ( T100293: [EPIC] Run browser tests against every MobileFrontend/Gather patch ).

@EBernhardson can you sync up with @Jdlrobson, @Jhernandez and @dduvall ?

hashar triaged this task as Medium priority.Aug 24 2015, 1:58 PM
zeljkofilipin set Security to None.
zeljkofilipin moved this task from Inbox to CI on the Browser-Tests-Infrastructure board.
Legoktm reopened this task as Open.Dec 4 2015, 1:40 AM
Legoktm added a subscriber: Legoktm.

The point of this task is to get *rid* of Cindy and use the proper CI infra.

Legoktm removed Deskana as the assignee of this task.Dec 4 2015, 1:40 AM
Deskana lowered the priority of this task from Medium to Lowest.Dec 4 2015, 1:42 AM

The point of this task is to get *rid* of Cindy and use the proper CI infra.

Good to know. I'll prioritise accordingly then.

Deskana moved this task from Needs triage to Search on the Discovery board.Dec 4 2015, 1:42 AM
Deskana renamed this task from Make browsertests for CirrusSearch run on every submitted patch to Make browsertests for CirrusSearch run on every submitted patch with proper CI infrastructure rather than a bot.Dec 4 2015, 1:45 AM
Legoktm raised the priority of this task from Lowest to Medium.Dec 4 2015, 6:09 AM

This isn't lowest priority.

This isn't lowest priority.

Then I'll have to take it out of Discovery, because it's pretty far down our list of things to fix.

Currently Cindy has special setup code for these test, that will need to be adapted (will be changed to not break other tests in T120360).

Deskana moved this task from Uncategorised to Technical on the CirrusSearch board.Dec 31 2015, 5:05 AM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptSep 15 2016, 11:36 AM
zeljkofilipin lowered the priority of this task from Medium to Low.May 29 2017, 10:29 AM
zeljkofilipin closed this task as Declined.May 29 2017, 10:43 AM

Unlikely to ever happen because of T139740: Port Selenium tests from Ruby to Node.js. Please reopen if you would like to make it happen.