Page MenuHomePhabricator

[EPIC] Run browser tests against every patch
Closed, DuplicatePublic

Description

As a Mobile Web developer I want to run a suite of browser tests against a patch so that I can feel (more) confident about merging it.

The Readers-Web-Backlog folk would like to run a suite of browser tests – strictly speaking I'd say we're talking about integration tests – against every submitted patch.

Questions
  • Is this possible given our existing infrastructure?
  • How do other companies get it done?
  • Why browser tests? Could we build a PHPUnit test harness that allows us to write controller (ish) tests?

These should probably be converted into spikes.

Event Timeline

phuedx created this task.May 20 2015, 3:45 PM
phuedx raised the priority of this task from to Needs Triage.
phuedx updated the task description. (Show Details)
phuedx added subscribers: phuedx, hashar, Jdlrobson.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 20 2015, 3:45 PM
phuedx updated the task description. (Show Details)May 20 2015, 3:45 PM
phuedx set Security to None.
phuedx updated the task description. (Show Details)
hashar added a subscriber: hashar.May 21 2015, 6:25 AM

We experimented them during winter 2013-2014 and were not very happy with the outcome. In short they take a while to run and most tests ends up exercise too much of the stack (and end up requiring CI to get the backend services installed)

Regardless, T55697 is the root task for it. Feel free to adjust its description and make it an epic :-}

In mobile we could explore categorising browser tests by ResourceLoader modules.
If we were to generate dependency trees for RL modules when ever changing a module we could run browser tests for all impacted modules. This might be smaller and easier to achieve then running all browser tests