While we still have a lot of work to do on nodepool and isolated CI, there are a few projects/teams that have small enough (sub)sets of MW-Selenium integration tests that experimenting with a Gerrit triggered JJB builder/template seems reasonable. The [[ https://lists.wikimedia.org/pipermail/qa/2015-June/002291.html | recent efforts ]] by @jdlrobson to set up an ad hoc CI bot for per-patch tests has proven that the set of tests can be small and refined enough to be stable in this use case.
We'll have to keep the set for each project lean- and mean-ish—as much as end-to-end style tests can be—using a convention of tags and/or Cucumber profiles. A lot of the hard work of setting up MediaWiki + extension + dependencies has been done for the QUnit jobs. Let's use that as a starting point.
We'll also need to make sure that the test user account can be created at runtime using the API helper in MW-Selenium—it might worthwhile to see if we can randomize/scope the user name to allow for stable parallelization within the same test suite.