Page MenuHomePhabricator
Create Task
Maniphest T231862

Selenium tests for Wikibase are being ran twice
Closed, ResolvedPublic
Actions

Assigned To
awight
Authored By
Ladsgroup
Sep 3 2019, 9:55 AM
Tags
Referenced Files
F30388290: wmf-quibble-timing-T231862.png
Sep 18 2019, 8:32 AM
F30209845: Screenshot_2019-09-03 quibble-vendor-mysql-hhvm-docker #69158 Console [Jenkins].png
Sep 3 2019, 10:14 AM
F30209844: Screenshot_2019-09-03 quibble-vendor-mysql-hhvm-docker #69158 Console [Jenkins](1).png
Sep 3 2019, 10:14 AM
Subscribers
Addshore
Aklapper
• alaa_wmde
awight
hashar
hoo
Jdforrester-WMF
View All 11 Subscribers
Tokens
"Love" token, awarded by Lucas_Werkmeister_WMDE."100" token, awarded by Pablo-WMDE.

Description

An example from

There are sections for Wikibase and data bridge selenium tests twice. Doing the exact same thing:

That's two more minutes in every tests.

Details

SubjectRepoBranchLines +/-
Deduplicate projectsintegration/quibblemaster+36 -16
jjb: Upgrade all quibble jobs to 0.0.35integration/configmaster+22 -22
docker: rebuild for quibble 0.0.35integration/configmaster+54 -0
Reduce side-effectsintegration/quibblemaster+58 -47
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ladsgroup created this task.Sep 3 2019, 9:55 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 3 2019, 9:55 AM
hashar added a project: Quibble.Sep 3 2019, 10:02 AM
hashar added a subscriber: awight.

Sounds like a bug in Quibble

Lucas_Werkmeister_WMDE added a comment.Sep 3 2019, 10:14 AM

Since the logs will be gone eventually, here’s two screenshots for posterity:

Screenshot_2019-09-03 quibble-vendor-mysql-hhvm-docker #69158 Console [Jenkins].png (1×1 px, 217 KB) Screenshot_2019-09-03 quibble-vendor-mysql-hhvm-docker #69158 Console [Jenkins](1).png (1×1 px, 216 KB)

awight added a comment.Sep 3 2019, 10:51 AM

Wikibase appears twice in this list:

00:02:15.523 DEBUG:quibble.cmd:Browser tests using DISPLAY=Xvfb, for projects mediawiki/extensions/Wikibase, mediawiki/core, mediawiki/skins/Vector, mediawiki/vendor, mediawiki/skins/MinervaNeue, mediawiki/skins/Vector, mediawiki/extensions/AbuseFilter, mediawiki/extensions/AntiSpoof, mediawiki/extensions/Babel, mediawiki/extensions/CheckUser, mediawiki/extensions/CirrusSearch, mediawiki/extensions/Cite, mediawiki/extensions/CiteThisPage, mediawiki/extensions/CodeEditor, mediawiki/extensions/ConfirmEdit, mediawiki/extensions/ContentTranslation, mediawiki/extensions/Echo, mediawiki/extensions/Elastica, mediawiki/extensions/EventLogging, mediawiki/extensions/FileImporter, mediawiki/extensions/Flow, mediawiki/extensions/Gadgets, mediawiki/extensions/GeoData, mediawiki/extensions/GlobalCssJs, mediawiki/extensions/GlobalPreferences, mediawiki/extensions/GuidedTour, mediawiki/extensions/ImageMap, mediawiki/extensions/InputBox, mediawiki/extensions/Interwiki, mediawiki/extensions/JsonConfig, mediawiki/extensions/MobileApp, mediawiki/extensions/MobileFrontend, mediawiki/extensions/NavigationTiming, mediawiki/extensions/ParserFunctions, mediawiki/extensions/PdfHandler, mediawiki/extensions/Poem, mediawiki/extensions/SandboxLink, mediawiki/extensions/SiteMatrix, mediawiki/extensions/SpamBlacklist, mediawiki/extensions/TemplateData, mediawiki/extensions/Thanks, mediawiki/extensions/TimedMediaHandler, mediawiki/extensions/Translate, mediawiki/extensions/UniversalLanguageSelector, mediawiki/extensions/VisualEditor, mediawiki/extensions/WikiEditor, mediawiki/extensions/Wikibase, mediawiki/extensions/WikibaseCirrusSearch, mediawiki/extensions/WikibaseMediaInfo, mediawiki/extensions/cldr

Vector also appears twice.

This should be a trivial bug to fix, it will be in cmd.py, function repos_to_clone.

gerritbot added a comment.Sep 3 2019, 10:56 AM

Change 520240 had a related patch set uploaded (by Awight; owner: Awight):
[integration/quibble@master] Reduce side-effects

https://gerrit.wikimedia.org/r/520240

gerritbot added a project: Patch-For-Review.Sep 3 2019, 10:56 AM
gerritbot added a comment.Sep 3 2019, 11:18 AM

Change 534134 had a related patch set uploaded (by Awight; owner: Awight):
[integration/quibble@master] Deduplicate projects

https://gerrit.wikimedia.org/r/534134

awight claimed this task.Sep 3 2019, 11:18 AM
awight moved this task from Backlog to In progress on the Continuous-Integration-Config board.
awight moved this task from Backlog to In progress on the Quibble board.

Ready for review.

• Pablo-WMDE subscribed.Sep 3 2019, 2:43 PM
hashar added a comment.Sep 3 2019, 5:56 PM

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

Ladsgroup added a comment.Sep 4 2019, 9:08 AM
In T231862#5462556, @hashar wrote:

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

That's sorta expected. WikibaseLexeme needs Wikibase to run its tests OTOH, wikibase while doesn't need wikibaselexeme but we need to keep it in the CI to avoid breaking lexeme by making mistakes in wikibase itself (saved us a couple of times)

awight added a comment.Sep 4 2019, 11:56 AM
In T231862#5464240, @Ladsgroup wrote:
In T231862#5462556, @hashar wrote:

The change was for Wikibase and it get added again in EXT_DEPENDENCIES. For example Wikibase depends on WikibaseLexeme which itself is flagged to depend on .. Wikibase :]

That's sorta expected. WikibaseLexeme needs Wikibase to run its tests OTOH, wikibase while doesn't need wikibaselexeme but we need to keep it in the CI to avoid breaking lexeme by making mistakes in wikibase itself (saved us a couple of times)

Quibble should be robust against this and I've written a patch to deduplicate the list of repos. However, what you describe is redudant with gate-and-submit, maybe you can take the lexeme dependency out of Wikibase and rely on the gated run to check for interactions?

Ladsgroup mentioned this in T232626: CI mediawiki/core run times have increased since July 26th, 2019.Sep 11 2019, 6:01 PM
Krinkle mentioned this in T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min.Sep 11 2019, 6:42 PM
hashar triaged this task as High priority.Sep 11 2019, 9:09 PM
hashar added a parent task: T232626: CI mediawiki/core run times have increased since July 26th, 2019.
gerritbot added a comment.Sep 11 2019, 9:23 PM

Change 520240 merged by jenkins-bot:
[integration/quibble@master] Reduce side-effects

https://gerrit.wikimedia.org/r/520240

gerritbot added a comment.Sep 11 2019, 9:23 PM

Change 534134 merged by jenkins-bot:
[integration/quibble@master] Deduplicate projects

https://gerrit.wikimedia.org/r/534134

Jdforrester-WMF added a comment.Sep 11 2019, 9:38 PM

Assuming the change to quibble works as expected, this should be fixed by minting the next version of quibble and updating CI jobs to use it.

Maintenance_bot removed a project: Patch-For-Review.Sep 11 2019, 10:10 PM
awight mentioned this in rQUIBBLE80ea5c277508: Reduce side-effects.Sep 11 2019, 10:49 PM
awight mentioned this in rQUIBBLE4d155ac79ebc: Deduplicate projects.
Stashbot added a comment.Sep 17 2019, 3:32 PM

Mentioned in SAL (#wikimedia-releng) [2019-09-17T15:32:07Z] <hashar> Tagging quibble 0.0.35 48f6b1d8db9eed8572f255fe86f920753149153c # T225730 T231862 T219694

Stashbot mentioned this in T219694: Enable compression for MW web responses in Jenkins jobs (e.g. Quibble, Fresnel).Sep 17 2019, 3:32 PM
gerritbot added a comment.Sep 17 2019, 3:38 PM

Change 537469 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: rebuild for quibble 0.0.35

https://gerrit.wikimedia.org/r/537469

gerritbot added a project: Patch-For-Review.Sep 17 2019, 3:38 PM
gerritbot added a comment.Sep 17 2019, 3:42 PM

Change 537469 merged by jenkins-bot:
[integration/config@master] docker: rebuild for quibble 0.0.35

https://gerrit.wikimedia.org/r/537469

Maintenance_bot removed a project: Patch-For-Review.Sep 17 2019, 4:10 PM
gerritbot added a comment.Sep 17 2019, 4:38 PM

Change 537485 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35

https://gerrit.wikimedia.org/r/537485

gerritbot added a project: Patch-For-Review.Sep 17 2019, 4:38 PM
gerritbot added a comment.Sep 17 2019, 6:37 PM

Change 537485 merged by jenkins-bot:
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35

https://gerrit.wikimedia.org/r/537485

Jdforrester-WMF closed this task as Resolved.Sep 17 2019, 6:40 PM
Jdforrester-WMF removed a project: Patch-For-Review.

Confirmed fixed – https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php72-docker/8663/consoleFull ran:

  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium mediawiki-core
  • Selenium skins/MinervaNeue
  • Selenium extensions/Cite
  • Selenium extensions/ContentTranslation
  • Selenium extensions/Echo
  • Selenium extensions/MobileFrontend
  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium extensions/WikibaseCirrusSearch

… whereas https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php72-docker/8692/consoleFull ran only:

  • Selenium extensions/Wikibase
  • Selenium extensions/Wikibase/client/data-bridge
  • Selenium mediawiki-core
  • Selenium extensions/Cite
  • Selenium extensions/ContentTranslation
  • Selenium extensions/Echo
  • Selenium extensions/MobileFrontend
  • Selenium extensions/WikibaseCirrusSearch
  • Selenium skins/MinervaNeue

(no duplicate Wikibase or data-bridge test runs).

hashar added a comment.Sep 18 2019, 7:51 AM

Released and deployed on September 17th with Quibble 0.0.35.

• Pablo-WMDE awarded a token.Sep 18 2019, 8:14 AM
hashar added a comment.Sep 18 2019, 8:32 AM

Looking at wmf-quibble-core-vendor-mysql-php72-docker build timing ( https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/buildTimeTrend ):

wmf-quibble-timing-T231862.png (400×500 px, 23 KB)

That went from ~ 22 minutes to ~ 18 minutes.

awight added a comment.Sep 18 2019, 8:40 AM

/p\ Imagine how much more we could improve by running the browser tests in parallel, even at per-repo granularity.

Lucas_Werkmeister_WMDE awarded a token.Sep 18 2019, 10:38 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL