Page MenuHomePhabricator

browsertests: triggers for ULS
Closed, ResolvedPublic

Description

This makes sense for ULS because they have a reasonable number of merges in a day, see https://gerrit.wikimedia.org/r/#/q/status:merged+project:mediawiki/extensions/UniversalLanguageSelector,n,z

This would entail:

  • Put the ULS tests into their own Jenkins build, and possibly into their own code repo the way Mobile is
  • Have Jenkins kick off the build of ULS tests targeting beta labs upon code merged in the ULS branch
  • Report the build status after each run

This came up because we found Bug 52115 in a timely way.


Version: unspecified
Severity: enhancement
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=52890
https://bugzilla.wikimedia.org/show_bug.cgi?id=53691
https://bugzilla.wikimedia.org/show_bug.cgi?id=57560

Details

Reference
bz52120

Event Timeline

bzimport raised the priority of this task from to High.Nov 22 2014, 1:53 AM
bzimport set Reference to bz52120.

Is this still something we want to do? ULS tests are now in ULS repo, so this should be doable. The only problem is that ULS tests are not stable enough:

https://wmf.ci.cloudbees.com/view/r-uls/

What about running the tests on patch submission?

Have added bug 53691 for the same for the VisualEditor repo.

Moving this under Continuous Integration radar.

Following a pair session with Željko, the tests can be flagged with a tag which we can then exclude when running tests. For ULS, I have introduced the tag @specific-settings which let us skip any tests that are not going to pass on a fresh wiki installation: https://gerrit.wikimedia.org/r/#/c/97487/

The browser tests manage to pass with above change:

https://integration.wikimedia.org/ci/job/mwext-browsertests-UniversalLanguageSelector-phantomjs/27/consoleFull

With a nice HTML report:
https://integration.wikimedia.org/ci/job/mwext-browsertests-UniversalLanguageSelector-phantomjs/27/artifact/report.html

Whenever the change in comment #5 is merged in, I will validate with i18n team how to get it triggered by Zuul.

Change 97741 had a related patch set uploaded by Hashar:
ULS browsertests on check pipeline

https://gerrit.wikimedia.org/r/97741

Change 97741 merged by jenkins-bot:
ULS browsertests on check pipeline

https://gerrit.wikimedia.org/r/97741

The trigger has been added in Zuul with https://gerrit.wikimedia.org/r/97741

The job will success whenever ULS change https://gerrit.wikimedia.org/r/#/c/97487/ is merged in.

hashar added a comment.Dec 6 2013, 1:56 PM

The job got fixed yesterday and browser tests are passing right now. It is not blocking changes yet though.