Page MenuHomePhabricator

[EPIC] trigger browser tests from Gerrit (tracking)
Closed, ResolvedPublic

Description

The VisualEditor and ULS MediaWiki extensions have recently been blessed with QA browser tests. We would need them to be in the Contint Jenkins and triggered by Zuul whenever a patchset is submitted as well as in the gate-and-submit pipeline.

That needs several step:

  • describe the jobs in Jenkins
  • add the triggers in Zuul configuration

Version: wmf-deployment
Severity: enhancement

Details

Reference
bz53697

Related Objects

StatusAssignedTask
Resolved zeljkofilipin
Resolved zeljkofilipin
ResolvedNone
Resolved zeljkofilipin
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
Resolvedhashar
ResolvedNone
Resolvedhashar
Resolvedhashar
DeclinedNone
ResolvedNone
DeclinedNone
Declinedhashar
DeclinedNone
Resolveddduvall
Resolveddduvall
OpenNone
ResolvedLegoktm
ResolvedJhernandez
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
Duplicatephuedx
ResolvedJdlrobson
ResolvedJdlrobson
DuplicateNone
ResolvedJdlrobson
ResolvedJdlrobson
ResolvedJdlrobson
DeclinedNone
InvalidNone
Resolvedbmansurov
Resolved rmoen
Resolved rmoen
OpenNone
Resolvedbmansurov
OpenNone
Resolved zeljkofilipin
ResolvedJdlrobson
ResolvedJdlrobson
Resolved zeljkofilipin
Resolved zeljkofilipin
Resolved zeljkofilipin
ResolvedNone
Resolvedhashar
DeclinedNone

Event Timeline

bzimport raised the priority of this task from to Normal.Nov 22 2014, 1:45 AM
bzimport set Reference to bz53697.
bzimport added a subscriber: Unknown Object (MLST).
hashar created this task.Sep 3 2013, 7:55 AM
hashar added a comment.Sep 3 2013, 4:22 PM

raise priority, add 'tracking' in summary.

hashar added a comment.Sep 3 2013, 4:25 PM

We will go with ULS first since we have a good experience interacting with the i18n team and they are in European timezone just like Zeljko and I.

The Jenkins jobo template in https://gerrit.wikimedia.org/r/#/c/86868/ would let us trigger tests.

Quick status update:

  • ULS has browser tests being triggered albeit lot of tests ends up being
  • MobileFrontend has a patch that has browser tests passing locally against a freshly installed wiki. Pending review / merge then I will add the browser tests.
  • VisualEditor browser tests depends on some work to be done to safely start and stop a Parsoid daemon for each test.

I am focusing this week on integrating Parsoid daemon in Jenkins

This is no more a top priority. We have most browsertests run twice per day instead.

We will eventually managed to get them added in Gerrit, but there is a bit more work that needs to be done first.

(In reply to Antoine "hashar" Musso from comment #6)

This is no more a top priority. We have most browsertests run twice per day
instead.
We will eventually managed to get them added in Gerrit, but there is a bit
more work that needs to be done first.

What's the status of this? Are there any projects that have browser tests triggering on each change (even if non-voting)?

If not, can you outline the blockers?

(In reply to Matthew Flaschen from comment #7)
<snip>

What's the status of this? Are there any projects that have browser tests
triggering on each change (even if non-voting)?
If not, can you outline the blockers?

It is not hold, mostly because we lack resources to do the integration / make them faster. On top of my head:

  • make them faster
  • runnable in parallel and aggregate the results
  • rethink the way we setup the environment to run the browser tests, it is currently a huge shell snippet which is copy pasted in different place

Feel free to take the leadership. Zeljkof and Nik Everett would be able to assist. I can offer reviews / guidances as well.

demon removed a subscriber: demon.Dec 16 2014, 6:05 PM
greg set Security to None.

Per a discussion we had in the Releng weekly checkin, triggering browser tests from Gerrit would need the disposable VMs. We would also need to setup MediaWiki and the required backend services such as Parsoid (and more as needed).

I have closed a few tasks that were requesting browser tests to be triggered on patch proposal. There is no point in keeping them open until we are actually ready to trigger such jobs.

hashar renamed this task from [project] trigger browser tests from Gerrit (tracking) to [EPIC] trigger browser tests from Gerrit (tracking).Jun 5 2015, 10:06 PM
hashar lowered the priority of this task from Normal to Low.
hashar added a project: Epic.
Jdlrobson changed the status of subtask T59560: browsertests: triggers for MobileFrontend from Declined to Resolved.Aug 20 2015, 4:39 AM

The description is a little confusing. Does VisualEditor do this already? I can't see the selenium job on a per commit basis....

The description is a little confusing. Does VisualEditor do this already? I can't see the selenium job on a per commit basis....

Still interested in the answer to this.

For mediawiki/extensions/VisualEditor no browser tests are run for patch upload nor merge, so it probably still needs to be done, possibly also for a few other closed blockers.

greg moved this task from INBOX to Epics on the Release-Engineering-Team board.Mar 11 2016, 10:09 PM
Krinkle removed a subscriber: Krinkle.Jun 2 2016, 8:01 PM
hashar added a subscriber: dduvall.

@dduvall made it possible to trigger selenium tests when a new patchset is triggered which unblocked a few tasks.

Originally we had this blocked on T47499 which comes from Bugzilla and was the parent tracking task / epic for the Continuous-Integration-Scaling project. It has progressed and we are not so far from having the jobs run on Nodepool. That is T137112: migrate mwext-mw-selenium to Nodepool instances which is more specific than the legacy epic task and better reflect the intention.

Change 341506 had a related patch set uploaded (by hashar):
[integration/config] Add experimental mwext-mw-selenium-jessie

https://gerrit.wikimedia.org/r/341506

Change 341506 merged by jenkins-bot:
[integration/config] Add experimental mwext-mw-selenium-jessie

https://gerrit.wikimedia.org/r/341506

hashar added a comment.Mar 7 2017, 9:38 AM

Have to make the following extension to trigger the job on patchset proposal:

CentralAuth
CentralNotice
Math
PageTriage
WikiLove

zeljkofilipin closed this task as Resolved.May 29 2017, 10:39 AM
zeljkofilipin claimed this task.

As far as I know this is resolved a long time ago. Please reopen if there is anything left to be done.