Page MenuHomePhabricator

Make browsertests for CirrusSearch run on every submitted patch with proper CI infrastructure rather than a bot
Closed, DeclinedPublic

Description

We have recently done a bit of refactoring to the browser tests for cirrussearch to get the time they take to run under control. Using parallel_cucumber with 4 processes they are down to around 5 minutes (or more depending on load of the server tests are running against).

This should initially be non-voting, but will be made to vote once we see that it doesn't have intermittent failures and such.

Event Timeline

EBernhardson raised the priority of this task from to Needs Triage.
EBernhardson updated the task description. (Show Details)
EBernhardson added a subscriber: EBernhardson.

I should note that while they are called browsertests since they are in cucumber, the majority of the tests now talk directly to the api and do not use the browser. If it makes things easier we can run only the @api tag and skip saucelabs,

Running just the api tests (bundle exec parallel_cucumber features/ -o '--tags @api) in four processes against a 4 core labs instance does 394 scenarios in ~4 minutes.
Running the entire suite (bundle exec parallel_cucumber features/) in four processes against a 4 core labs instance does 502 scenarios in ~5.5 minutes

hashar triaged this task as Medium priority.Aug 24 2015, 1:58 PM
Legoktm added a subscriber: Legoktm.

The point of this task is to get *rid* of Cindy and use the proper CI infra.

Deskana lowered the priority of this task from Medium to Lowest.Dec 4 2015, 1:42 AM

The point of this task is to get *rid* of Cindy and use the proper CI infra.

Good to know. I'll prioritise accordingly then.

Deskana renamed this task from Make browsertests for CirrusSearch run on every submitted patch to Make browsertests for CirrusSearch run on every submitted patch with proper CI infrastructure rather than a bot.Dec 4 2015, 1:45 AM
Legoktm raised the priority of this task from Lowest to Medium.Dec 4 2015, 6:09 AM

This isn't lowest priority.

This isn't lowest priority.

Then I'll have to take it out of Discovery, because it's pretty far down our list of things to fix.

Currently Cindy has special setup code for these test, that will need to be adapted (will be changed to not break other tests in T120360).

zeljkofilipin lowered the priority of this task from Medium to Low.May 29 2017, 10:29 AM

Unlikely to ever happen because of T139740: Port Selenium tests from Ruby to Node.js. Please reopen if you would like to make it happen.