Maniphest T200976

Wikibase CI: wmf-quibble-vendor-mysql-hhvm-docker job should include Scribunto
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	WMDE-leszek
	Aug 2 2018, 7:30 AM

Description

As discovered when investigating T200693 it turned out that apparently quibble job for Wikibase does not pull in Scribunto extension, which results in quite some tests not being run.

We might have missed the test failure that potentially quibble job could have spotted.

I cannot at least Scribunto listed in EXT_DEPENDENCIES in https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/2192/parameters/. With quick look I failed a place in integration/config repo where I could add Scribunto there, thus the ticket.

Details

	Subject	Repo	Branch	Lines +/-
	Added scribunto to shared gate quibble job	integration/config	master	+1 -0

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Resolved	daniel	T204065 WikibaseLexeme CI broken (database errors)
Open	None	T249674 Have all Wikimedia production extensions and skins in the CI gate
Resolved	Addshore	T204153 Add more Wikibase extensions to gatedextensions
Open	None	T204252 Have dependencies of gated extensions in the gate
Duplicate	None	T200976 Wikibase CI: wmf-quibble-vendor-mysql-hhvm-docker job should include Scribunto
Resolved	Jdforrester-WMF	T125050 [Task] Add Scribunto to extension-gate in CI
Resolved	Jdforrester-WMF	T126670 Have CI set `$wgScribuntoDefaultEngine = 'luasandbox` to speed up parser tests
Resolved	None	T128191 Wikibase Scribunto tests are failing, with the tests depending on configuration
Resolved	hoo	T128245 Use $this->getEngine() instead of Scribunto::newDefaultEngine()
Resolved	• chasemp	T162166 For contintcloud either add RAM or Swap to the instances
Resolved	Jdforrester-WMF	T225068 Add a PHPUnit group to skip test on gated CI runs

Event Timeline

WMDE-leszek created this task.Aug 2 2018, 7:30 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 2 2018, 7:30 AM

With quick look I failed a place in integration/config repo where I could add Scribunto there, thus the ticket.

gatedextensions list in zuul/parameter_functions.py. https://www.mediawiki.org/wiki/Continuous_integration/Tutorials/Adding_your_extension_to_the_shared_gate_job is uh, non-existent :(

That setting I've actually found. But, as you say, it is, well, shared. I thought I could just add it for Wikibase (but that would mean having a quibble job for Wikibase and not using wmf-quibble-vendor-mysql-hhvm-docker job).
Let me add it to it and let's see if @hashar objects.

Change 449949 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[integration/config@master] Added scribunto to shared gate quibble job

https://gerrit.wikimedia.org/r/449949

gerritbot added a project: Patch-For-Review.Aug 2 2018, 7:58 AM

Scribunto is intentionally not in the list of gated extensions. Its tests take a while and would be run for any patch made on any of the extension, even when we well know the tests are not needed.

T125050 has some history about a previous attempt.

We need CI to be smarter by making run solely:

run the test of the extension for which a patch got sent
for the other extensions, only run a subset of tests (integration tests)

Right now, any patch on any of the gated extension also run all the Wikibase tests (as well as all tests from all extensions). That partly explains why the job is so slow.

(note I recently phased out the 'mediawiki-extensions-*' jobs in favor of 'wmf-quibble-*' jobs which use Quibble)

Let's create a second gate job that runs in parallel? Scribunto is an important enough extension that we should be regularly running its tests.

Just to clarify: the Scribunto tests do run for patches sent to Scribunto. It is just that when sending a patch to Wikibase, its integration tests with Scribunto are skip since the extension is not present.

We would need all our test suites (PHPUnit, QUnit, Selenium) to be smarter. Timo wrote a nice summary for Selenium tests at T199939#4442512 which would be applicable to others suites.

So essentially I would like to hold on this task until CI / test suites are ready.

hashar mentioned this in T199939: Audit tests/selenium/LocalSettings.php file aiming at possibly deprecating the feature.Aug 3 2018, 9:12 AM

WMDE-leszek mentioned this in T201496: Wikibase CI: Quibble job should possibly include Math extension.Aug 8 2018, 10:18 AM

hashar mentioned this in T204153: Add more Wikibase extensions to gatedextensions.Sep 12 2018, 7:32 PM

hashar added a subtask: T125050: [Task] Add Scribunto to extension-gate in CI.Sep 13 2018, 3:35 PM

hashar mentioned this in T204252: Have dependencies of gated extensions in the gate.

greg added a parent task: T204252: Have dependencies of gated extensions in the gate.Nov 21 2018, 12:21 AM

Addshore mentioned this in T218702: ParserIntegrationTest for Scribunto failing in Wikibase CI.Mar 19 2019, 4:48 PM

hashar closed this task as a duplicate of T125050: [Task] Add Scribunto to extension-gate in CI.Apr 17 2019, 8:24 PM

Change 449949 abandoned by Jforrester:
Added scribunto to shared gate quibble job

Reason:
Duplicate of Ia128e3abe7 (which also isn't merged).

https://gerrit.wikimedia.org/r/449949