Page MenuHomePhabricator

Make browser tests voting for all repos of WMF deployed code
Closed, ResolvedPublic

Description

This is blocked by T55697: [EPIC] trigger browser tests from Gerrit (tracking)

After that (having the ability to trigger the tests) we should make them voting. This will probably take a while to remove known bad tests, determine how to deal with flakey ones (eg: only fail the build after 3 failures in a row?), and get buy-in from effected teams.

Related Objects

StatusAssignedTask
Resolvedzeljkofilipin
Resolvedzeljkofilipin
ResolvedNone
Resolvedzeljkofilipin
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
ResolvedNone
Resolvedhashar
DeclinedNone
ResolvedNone
DeclinedNone
Declinedhashar
DeclinedNone
Resolved dduvall
Resolved dduvall
OpenNone
ResolvedLegoktm
ResolvedJhernandez
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
Duplicatephuedx
ResolvedJdlrobson
ResolvedJdlrobson
DuplicateNone
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
DeclinedNone
InvalidNone
Resolvedbmansurov
Resolved rmoen
Resolved rmoen
OpenNone
Resolvedbmansurov
OpenNone
Resolvedzeljkofilipin
ResolvedJdlrobson
ResolvedJdlrobson
Resolvedzeljkofilipin
Resolvedzeljkofilipin
Resolvedzeljkofilipin
ResolvedNone
Resolvedhashar
DeclinedNone

Event Timeline

greg created this task.Mar 5 2015, 6:19 PM
greg raised the priority of this task from to Normal.
greg updated the task description. (Show Details)
greg added subscribers: greg, hashar, zeljkofilipin and 2 others.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 5 2015, 6:19 PM

Test suites need to be faster as well. Once we work out ways to increase test isolation (for instance T90964) we should also work towards making them execute in parallel.

Krinkle added a subscriber: Krinkle.EditedMar 6 2015, 6:51 PM

Test suites need to be faster as well. Once we work out ways to increase test isolation (for instance T90964) we should also work towards making them execute in parallel.

The jobs that run on-commit (as opposed to from cron) should not in any way connect to beta labs, staging or other wikis outside the jenkins slave.

Test suites need to be faster as well. Once we work out ways to increase test isolation (for instance T90964) we should also work towards making them execute in parallel.

The jobs that run on-commit (as opposed to from cron) should not in any way connect to beta labs, staging or other wikis outside the jenkins slave.

I should have been more clear: Whatever method/environment we use to isolate tests needs to support concurrency within a single browser test suite. I'm not sure that a one-mediawiki-installation-to-one-test-process setup is going to accommodate enough concurrency. You're right, though. It can't be performed on a wiki that's shared beyond the workspace and, since it's an isolated environment, users and other resources might as well be provisioned in a setup/teardown fashion instead of relying on pre-provisioned resource pools.

zeljkofilipin lowered the priority of this task from Normal to Low.May 29 2017, 10:28 AM

@greg do we still need this task or is now replaced by release pipeline?

greg moved this task from INBOX to Epics on the Release-Engineering-Team board.May 30 2017, 10:45 PM
zeljkofilipin closed this task as Resolved.Nov 7 2017, 3:19 PM
zeljkofilipin claimed this task.

This is implemented in Node.js Selenium framework (T139740). For all repos that have browser tests (core, RelatedArticles, Popups...) all tests are running for every patch set.