create a Wikibase browser test job running against a fresh MediaWiki installation
Closed, ResolvedPublic

Description

create a Wikibase browser test job running against a fresh MediaWiki installation
This requires that the browsertests are in the same repo as the extension under tests/browser/. The Gemfile needs to be at the root of the extension repo.
See also: https://www.mediawiki.org/wiki/Continuous_integration/Browser_tests

Change 252240 had a related patch set uploaded (by JanZerebecki):
add composer version of mwext-mw-selenium

https://gerrit.wikimedia.org/r/252240

Change 252240 merged by jenkins-bot:
add composer version of mwext-mw-selenium

https://gerrit.wikimedia.org/r/252240

JanZerebecki set Security to None.
JanZerebecki removed a project: Patch-For-Review.

Patch in wikibase that adds an initial browsertest: https://gerrit.wikimedia.org/r/#/c/253324/

Change 253636 had a related patch set uploaded (by JanZerebecki):
Add set_ext_dependencies to mwext-mw-selenium-composer

https://gerrit.wikimedia.org/r/253636

Change 253636 merged by jenkins-bot:
Add set_ext_dependencies to mwext-mw-selenium-composer

https://gerrit.wikimedia.org/r/253636

Change 253324 had a related patch set uploaded (by JanZerebecki):
add browsertest to be run by jenkins

https://gerrit.wikimedia.org/r/253324

JanZerebecki claimed this task.

I don't think we should do this. The browser tests are not for Wikibase, but for Wikidata, that's why we moved them. If we would have an actual Wikidata extension, they could be there, and that would make a lot of sense for me.

We can still run the browser tests in Wikibase.git against beta or test.wikidata.org. Why shouldn't the browser tests that test repo wikibase and work on a one wiki repo installation reside in Wikibase.git? (To clarify I'm not suggesting to move anything else to Wikibase.git.) How else do you suggest to run browser tests before merge (that is what this job can do). Or are you suggesting we should never run them automatically before merge?

daniel added a subscriber: daniel.Nov 30 2015, 11:49 AM

@adrianheine I think the point is that we should have browsertests for wikibase as such, and they should live in the wikibase extension, and should automatically be executed by CI. The wikidata browser tests should be separate. Running those via CI too would probably be nice, but I'm not sure how to do it best.

Change 253324 merged by jenkins-bot:
add browsertest to be run by jenkins

https://gerrit.wikimedia.org/r/253324

You could at least ask the people that did the reverse move not even a year ago.

@adrianheine It's not that we are moving the tests back. This is about setting up infrastructure for running browsertests via CI. Which tests will actually be in the Wikibase repo is a different question, to be decided on a case-by-case basis.

The idea is that we always write browser tests along with user facing features, just like we always write unit tests. Having the tests in the same repo is essential for that.

Enabled the job in abd24ff0ef118188d22f6c130c5a718ef4cd4ac3 .
@Tobi_WMDE_SW Wanna add anything regarding T118284#1840987 ?
Otherwise this is done.

adrianheine closed this task as "Resolved".Dec 7 2015, 10:19 AM
adrianheine moved this task from Review to Done on the Wikidata-Sprint-2015-12-01 board.

@adrianheine @JanZerebecki I love to see movement in the direction of having a browsertest job running commit-wise, but I don't have an answer on the question whether it is good to move the tests to the Wikibase.git repository. One problem I see here is, that if you modify one of the JavaScript components, let's say ValueView, you would have to adjust the browser tests in Wikibase. Or, in case you break one of the tests with a change in ValueView, you would only notice the breakage on a change done in Wikibase.git (and in the worst case on an unrelated change in Wikibase.git).

One problem I see here is, that if you modify one of the JavaScript components, let's say ValueView, you would have to adjust the browser tests in Wikibase.

That is something that may happen, but that was already the case with the browser tests in a separate repository. Any such change needs a major release of ValueView. Wikibase should not automatically upgrade to new major releases. Thus the change in Wikibase to the new major release of ValueView can at the same time adapt the browser tests. Having more tests as Qunit tests in ValueView instead of browser tests would also minimize the amount of browser test changes.