Page MenuHomePhabricator

[GOAL]: Set up testing environments that run browser tests on all patches.
Closed, ResolvedPublic

Description

Integration testing infrastructure

Why

Integration testing helps us find regressions and test our code from top to
bottom, testing all parts of the stack. It's automated QA, and our properties
are very user exposed, so browser testing helps us.

Current situation

Tests run on master a couple times a day. Emails are sent to the mailing list.
Really flaky and a lot of noise. Developers don't pay much attention since it's
really hard to diagnose what is really broken and when did it broke.
Stakeholders regularly see failures without clear action being taken.

Objectives

Continuously run browser tests on every patch submitted on development. The
test run will notify in the patch that introduces the code if the tests passed
or not with the log.

  • Set up and document testing environment(s).
  • Set up and document patch triggers.
  • Set up and document test running, logging and submitting results.
  • Set up email reporting patches that broke the tests and where merged before the report came back.
  • Expose the instance log and dashboard.
Estimated work

2 sprints ~ 50%

Related Objects

StatusSubtypeAssignedTask
Resolvedzeljkofilipin
Resolvedzeljkofilipin
ResolvedJhernandez
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
Duplicatephuedx
ResolvedJdlrobson
ResolvedJdlrobson
DuplicateNone
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
DeclinedNone
InvalidNone
Resolvedbmansurov
Resolved rmoen
Resolved rmoen
OpenNone
Resolvedbmansurov

Event Timeline

Jhernandez claimed this task.
Jhernandez raised the priority of this task from to Needs Triage.
Jhernandez updated the task description. (Show Details)
Jhernandez added a subscriber: Jhernandez.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 30 2015, 11:19 AM
Jhernandez triaged this task as High priority.Jun 30 2015, 11:24 AM
Jhernandez set Security to None.
Jhernandez renamed this task from Set up testing environments that run browser tests on all patches. to [GOAL]: Set up testing environments that run browser tests on all patches..Jul 31 2015, 4:26 PM
Jdlrobson closed this task as Resolved.Sep 16 2015, 7:05 PM
Jdlrobson added a subscriber: Jdlrobson.

This goal has now been achieved. Great work guys!