Add @integration tests to Gather browser tests
Closed, ResolvedPublic
Actions

Description

Using as a guideline: https://gerrit.wikimedia.org/r/#/c/221313/ let's do the same for Gather.

Details

Project	Branch	Lines +/-	Subject
mediawiki/extensions/Gather	master	+17 -0	QA: Avoid flakiness in various test steps due to race conditions
integration/config	master	+3 -1	Run mw-selenium job in test and gate-and-submit for Gather
mediawiki/extensions/Gather	master	+9 -12	QA: More integration tests on each commit
mediawiki/extensions/Gather	master	+35 -21	Run Gather browser tests on various commits

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	zeljkofilipin	T91669 Make browser tests voting for all repos of WMF deployed code
Resolved	zeljkofilipin	T55697 [EPIC] trigger browser tests from Gerrit (tracking)
Resolved	Jhernandez	T104315 [GOAL]: Set up testing environments that run browser tests on all patches.
Resolved	Jdlrobson	T100293 [EPIC] Run browser tests against every MobileFrontend/Gather patch
Resolved	Jdlrobson	T107908 Add @integration tests to Gather browser tests
Declined	None	T89096 Figure out a system to override default settings when in test context
Invalid	None	T72250 Allow tests to specify what extensions and or what order things are loaded in

Event Timeline

Jdlrobson created this task.Aug 4 2015, 5:33 PM

Jdlrobson raised the priority of this task from to High.

Jdlrobson updated the task description. (Show Details)

Jdlrobson added projects: Reading-Web-Planning, Epic, Gather, MobileFrontend.

Jdlrobson added subscribers: dduvall, Jdlrobson, Legoktm and 3 others.

Restricted Application added a project: Readers-Web-Backlog. · View Herald TranscriptAug 4 2015, 5:33 PM

Jdlrobson moved this task from Backlog to Team: web on the MobileFrontend board.Aug 4 2015, 6:30 PM

KLans_WMF moved this task from Incoming to 2016-17 Q2 on the Readers-Web-Backlog board.Aug 5 2015, 5:14 PM

Jhernandez removed a project: Reading-Web-Planning.Aug 5 2015, 5:15 PM

Jhernandez set Security to None.

Jdlrobson updated the task description. (Show Details)Aug 5 2015, 9:00 PM

Jdlrobson mentioned this in T107587: Migrating mobile browser tests to JJB.Aug 12 2015, 12:30 AM

Jdlrobson removed projects: Readers-Web-Backlog, MobileFrontend, Epic.

Change 230953 had a related patch set uploaded (by Jdlrobson):
Run Gather browser tests on each commit

https://gerrit.wikimedia.org/r/230953

gerritbot added a project: Patch-For-Review.Aug 12 2015, 12:35 AM

Jdlrobson moved this task from Needs triage to Must haves on the Gather board.Aug 12 2015, 5:59 AM

Jdlrobson claimed this task.Aug 12 2015, 10:58 PM

Jdlrobson added a project: Reading-Web-Sprint-53-12-Monkeys.

Jdlrobson moved this task from Needs Analysis to Code Review on the Reading-Web-Sprint-53-12-Monkeys board.

phuedx moved this task from Code Review to -1 (Needs More Work) on the Reading-Web-Sprint-53-12-Monkeys board.Aug 13 2015, 2:32 PM

Jdlrobson moved this task from -1 (Needs More Work) to Doing on the Reading-Web-Sprint-53-12-Monkeys board.Aug 13 2015, 11:04 PM

Change 231468 had a related patch set uploaded (by Jdlrobson):
QA: More integration tests on each commit

https://gerrit.wikimedia.org/r/231468

Jdlrobson moved this task from Doing to Code Review on the Reading-Web-Sprint-53-12-Monkeys board.Aug 14 2015, 12:39 AM

Jdlrobson moved this task from Code Review to -1 (Needs More Work) on the Reading-Web-Sprint-53-12-Monkeys board.Aug 14 2015, 12:49 AM

Jdlrobson moved this task from -1 (Needs More Work) to Doing on the Reading-Web-Sprint-53-12-Monkeys board.Aug 14 2015, 5:10 PM

Jhernandez edited projects, added Reading-Web-Sprint-54-28-Days-Later; removed Reading-Web-Sprint-53-12-Monkeys.Aug 17 2015, 4:05 PM

Jhernandez moved this task from Needs Analysis to Doing on the Reading-Web-Sprint-54-28-Days-Later board.

Jdlrobson moved this task from Doing to Code Review on the Reading-Web-Sprint-54-28-Days-Later board.Aug 17 2015, 7:12 PM

It's barfing because $wgMFEnableBeta = false; by default.
We'll need to force $wgMFEnableBeta = true; on
With such a mechanism we'd also be able to fix half of T107910

https://gerrit.wikimedia.org/r/#/c/230953/10 should be passing but it only covers a few of our tests.
Have added blocking tasks for the other ones.

https://gerrit.wikimedia.org/r/#/c/230953/ has a +1 from baha but someone needs to be bold and merge it :)

@Jdlrobson: @bmansurov mentioned mentioning the mediawiki-selenium gem update. I can be bold.

Change 230953 merged by jenkins-bot:
Run Gather browser tests on various commits

https://gerrit.wikimedia.org/r/230953

Diffusion mentioned this in rMEXTceccaa301db2: Updated mediawiki/extensions Project: mediawiki/extensions/Gather….Aug 21 2015, 6:47 PM

Jdlrobson mentioned this in rEGAT001c89d6b7b2: Run Gather browser tests on various commits.Aug 21 2015, 6:47 PM

Change 231468 merged by jenkins-bot:
QA: More integration tests on each commit

https://gerrit.wikimedia.org/r/231468

phuedx mentioned this in rEGAT2185b6150193: QA: More integration tests on each commit.Aug 21 2015, 7:13 PM

Diffusion mentioned this in rMEXTf9f1a893022d: Updated mediawiki/extensions Project: mediawiki/extensions/Gather….Aug 21 2015, 7:13 PM

phuedx moved this task from Code Review to Ready for Signoff on the Reading-Web-Sprint-54-28-Days-Later board.Aug 21 2015, 7:19 PM

Jdlrobson moved this task from Ready for Signoff to Doing on the Reading-Web-Sprint-54-28-Days-Later board.Aug 21 2015, 7:35 PM

Needs further work. I should have noted that check experimental is needed to run the browser tests - they are currently not turned on a per commit basis.
https://integration.wikimedia.org/ci/job/mwext-mw-selenium/469/console

@dduvall I'm guessing I screwed up with my LocalSettings definition?

Hopefully https://gerrit.wikimedia.org/r/#/c/233032/ will do it... duhhhh

Jdlrobson moved this task from Doing to Code Review on the Reading-Web-Sprint-54-28-Days-Later board.Aug 21 2015, 7:49 PM

Change 233082 had a related patch set uploaded (by Jdlrobson):
Run selenium job in test and gate and submit

https://gerrit.wikimedia.org/r/233082

Change 233082 merged by jenkins-bot:
Run mw-selenium job in test and gate-and-submit for Gather

https://gerrit.wikimedia.org/r/233082

dduvall mentioned this in rCICFf87056377df8: Run mw-selenium job in test and gate-and-submit for Gather.Aug 22 2015, 12:07 AM

Now we should be good to go.
Checking https://gerrit.wikimedia.org/r/#/c/223309/ to see if it passes.

Forrestbot added a project: WMF-deploy-2015-08-25_(1.26wmf20).Aug 23 2015, 12:34 AM

Awesome to see this happen.

https://integration.wikimedia.org/ci/job/mwext-mw-selenium/486/console

Does this cover all of Gather's tests?

Jhernandez moved this task from Ready for Signoff to Done on the Reading-Web-Sprint-54-28-Days-Later board.Aug 24 2015, 9:20 AM

they cover the majority of them if not all of them but currently tests are failing (false positive) - this should fix it - https://gerrit.wikimedia.org/r/233425

Jdlrobson moved this task from Done to Ready for Signoff on the Reading-Web-Sprint-54-28-Days-Later board.Aug 24 2015, 4:01 PM

Change 233425 had a related patch set uploaded (by Jhernandez):
QA: Do not rely on browser url for login test

https://gerrit.wikimedia.org/r/233425

Moving back to -1 to get this fixed

Jhernandez moved this task from Ready for Signoff to -1 (Needs More Work) on the Reading-Web-Sprint-54-28-Days-Later board.Aug 25 2015, 9:55 AM

Jdlrobson moved this task from -1 (Needs More Work) to Code Review on the Reading-Web-Sprint-54-28-Days-Later board.Aug 25 2015, 10:34 PM

phuedx moved this task from Code Review to -1 (Needs More Work) on the Reading-Web-Sprint-54-28-Days-Later board.Aug 26 2015, 10:34 AM

Jdlrobson moved this task from -1 (Needs More Work) to Code Review on the Reading-Web-Sprint-54-28-Days-Later board.Aug 26 2015, 4:18 PM

Jdlrobson moved this task from Code Review to -1 (Needs More Work) on the Reading-Web-Sprint-54-28-Days-Later board.Aug 26 2015, 5:19 PM

Jdlrobson moved this task from -1 (Needs More Work) to Doing on the Reading-Web-Sprint-54-28-Days-Later board.Aug 26 2015, 7:04 PM