Page MenuHomePhabricator

Selenium tests for Wikibase are being ran twice
Closed, ResolvedPublic

Details

Related Gerrit Patches:
integration/quibble : masterDeduplicate projects
integration/config : masterjjb: Upgrade all quibble jobs to 0.0.35
integration/config : masterdocker: rebuild for quibble 0.0.35
integration/quibble : masterReduce side-effects

Event Timeline

Ladsgroup created this task.Sep 3 2019, 9:55 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 3 2019, 9:55 AM
hashar added a subscriber: awight.

Sounds like a bug in Quibble

Since the logs will be gone eventually, here’s two screenshots for posterity:

Wikibase appears twice in this list:

00:02:15.523 DEBUG:quibble.cmd:Browser tests using DISPLAY=Xvfb, for projects mediawiki/extensions/Wikibase, mediawiki/core, mediawiki/skins/Vector, mediawiki/vendor, mediawiki/skins/MinervaNeue, mediawiki/skins/Vector, mediawiki/extensions/AbuseFilter, mediawiki/extensions/AntiSpoof, mediawiki/extensions/Babel, mediawiki/extensions/CheckUser, mediawiki/extensions/CirrusSearch, mediawiki/extensions/Cite, mediawiki/extensions/CiteThisPage, mediawiki/extensions/CodeEditor, mediawiki/extensions/ConfirmEdit, mediawiki/extensions/ContentTranslation, mediawiki/extensions/Echo, mediawiki/extensions/Elastica, mediawiki/extensions/EventLogging, mediawiki/extensions/FileImporter, mediawiki/extensions/Flow, mediawiki/extensions/Gadgets, mediawiki/extensions/GeoData, mediawiki/extensions/GlobalCssJs, mediawiki/extensions/GlobalPreferences, mediawiki/extensions/GuidedTour, mediawiki/extensions/ImageMap, mediawiki/extensions/InputBox, mediawiki/extensions/Interwiki, mediawiki/extensions/JsonConfig, mediawiki/extensions/MobileApp, mediawiki/extensions/MobileFrontend, mediawiki/extensions/NavigationTiming, mediawiki/extensions/ParserFunctions, mediawiki/extensions/PdfHandler, mediawiki/extensions/Poem, mediawiki/extensions/SandboxLink, mediawiki/extensions/SiteMatrix, mediawiki/extensions/SpamBlacklist, mediawiki/extensions/TemplateData, mediawiki/extensions/Thanks, mediawiki/extensions/TimedMediaHandler, mediawiki/extensions/Translate, mediawiki/extensions/UniversalLanguageSelector, mediawiki/extensions/VisualEditor, mediawiki/extensions/WikiEditor, mediawiki/extensions/Wikibase, mediawiki/extensions/WikibaseCirrusSearch, mediawiki/extensions/WikibaseMediaInfo, mediawiki/extensions/cldr

Vector also appears twice.

This should be a trivial bug to fix, it will be in cmd.py, function repos_to_clone.

Change 520240 had a related patch set uploaded (by Awight; owner: Awight):
[integration/quibble@master] Reduce side-effects

https://gerrit.wikimedia.org/r/520240

Change 534134 had a related patch set uploaded (by Awight; owner: Awight):
[integration/quibble@master] Deduplicate projects

https://gerrit.wikimedia.org/r/534134

awight claimed this task.Sep 3 2019, 11:18 AM
awight moved this task from Backlog to In progress on the Continuous-Integration-Config board.
awight moved this task from Backlog to In progress on the Quibble board.

Ready for review.

hashar added a comment.Sep 3 2019, 5:56 PM

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

That's sorta expected. WikibaseLexeme needs Wikibase to run its tests OTOH, wikibase while doesn't need wikibaselexeme but we need to keep it in the CI to avoid breaking lexeme by making mistakes in wikibase itself (saved us a couple of times)

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

That's sorta expected. WikibaseLexeme needs Wikibase to run its tests OTOH, wikibase while doesn't need wikibaselexeme but we need to keep it in the CI to avoid breaking lexeme by making mistakes in wikibase itself (saved us a couple of times)

Quibble should be robust against this and I've written a patch to deduplicate the list of repos. However, what you describe is redudant with gate-and-submit, maybe you can take the lexeme dependency out of Wikibase and rely on the gated run to check for interactions?

Change 520240 merged by jenkins-bot:
[integration/quibble@master] Reduce side-effects

https://gerrit.wikimedia.org/r/520240

Change 534134 merged by jenkins-bot:
[integration/quibble@master] Deduplicate projects

https://gerrit.wikimedia.org/r/534134

Assuming the change to quibble works as expected, this should be fixed by minting the next version of quibble and updating CI jobs to use it.

Change 537469 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: rebuild for quibble 0.0.35

https://gerrit.wikimedia.org/r/537469

Change 537469 merged by jenkins-bot:
[integration/config@master] docker: rebuild for quibble 0.0.35

https://gerrit.wikimedia.org/r/537469

Change 537485 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35

https://gerrit.wikimedia.org/r/537485

Change 537485 merged by jenkins-bot:
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35

https://gerrit.wikimedia.org/r/537485

Jdforrester-WMF closed this task as Resolved.Sep 17 2019, 6:40 PM
Jdforrester-WMF removed a project: Patch-For-Review.

Confirmed fixed – https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php72-docker/8663/consoleFull ran:

  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium mediawiki-core
  • Selenium skins/MinervaNeue
  • Selenium extensions/Cite
  • Selenium extensions/ContentTranslation
  • Selenium extensions/Echo
  • Selenium extensions/MobileFrontend
  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium extensions/WikibaseCirrusSearch

… whereas https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php72-docker/8692/consoleFull ran only:

  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium mediawiki-core
  • Selenium extensions/Cite
  • Selenium extensions/ContentTranslation
  • Selenium extensions/Echo
  • Selenium extensions/MobileFrontend
  • Selenium extensions/WikibaseCirrusSearch
  • Selenium skins/MinervaNeue

(no duplicate Wikibase or data-bridge test runs).

Released and deployed on September 17th with Quibble 0.0.35.

Looking at wmf-quibble-core-vendor-mysql-php72-docker build timing ( https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/buildTimeTrend ):

That went from ~ 22 minutes to ~ 18 minutes.

/p\ Imagine how much more we could improve by running the browser tests in parallel, even at per-repo granularity.