Page MenuHomePhabricator
Create Task
Maniphest T193244

selenium-CirrusSearch-jessie does not run any tests
Closed, ResolvedPublic
Actions

Description

Since #97 selenium-CirrusSearch-jessie does not run any tests.

From #107:

00:02:13.115 + cd /home/jenkins/workspace/selenium-CirrusSearch-jessie/src
...
00:01:05.696 + ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/CirrusSearch/tests/selenium/specs/smoke_test.js
...
00:01:06.405 pattern /home/jenkins/workspace/selenium-CirrusSearch-jessie/src/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
00:01:06.406 pattern /home/jenkins/workspace/selenium-CirrusSearch-jessie/src/skins/*/tests/selenium/specs/**/*.js did not match any file

Done

🏝 444609 Selenium: create selenium-daily NPM script

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
selenium: Do not exclude CirrusSearch when running smoke testsintegration/configmaster+1 -1
selenium: Do not exclude CirrusSearch when running smoke testsmediawiki/coremaster+10 -0
Selenium: Create selenium-daily NPM scriptmediawiki/extensions/CirrusSearchmaster+91 -1
Customize query in gerrit

Related Objects
Search...

Event Timeline

zeljkofilipin created this task.Apr 27 2018, 1:26 PM
Restricted Application added projects: Discovery-ARCHIVED, Discovery-Search, Release-Engineering-Team (Kanban). · View Herald TranscriptApr 27 2018, 1:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
zeljkofilipin triaged this task as High priority.Apr 27 2018, 1:27 PM
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.
zeljkofilipin moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
zeljkofilipin added a parent task: T190994: Q4 Selenium framework improvements.
zeljkofilipin updated the task description. (Show Details)Apr 27 2018, 1:31 PM
zeljkofilipin added a comment.Apr 27 2018, 1:36 PM

#96 had a different problem, resolved by 427362

00:01:16.922 + ./node_modules/.bin/wdio tests/selenium/wdio.conf.jenkins.js --spec extensions/CirrusSearch/tests/selenium/specs/smoke_test.js
...
00:01:17.784 Failed loading configuration file: /home/jenkins/workspace/selenium-CirrusSearch-jessie/src/tests/selenium/wdio.conf.jenkins.js
zeljkofilipin updated the task description. (Show Details)Apr 27 2018, 1:38 PM
zeljkofilipin added a comment.Apr 27 2018, 1:58 PM

A couple of similar jobs are working just fine.

selenium-Popups-jessie/205/console

00:01:17.899 + cd /home/jenkins/workspace/selenium-Popups-jessie/src
...
00:01:17.899 + ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/Popups/tests/selenium/specs/previews.js
....
00:01:18.676 pattern /home/jenkins/workspace/selenium-Popups-jessie/src/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
00:01:18.677 pattern /home/jenkins/workspace/selenium-Popups-jessie/src/skins/*/tests/selenium/specs/**/*.js did not match any file
00:01:18.809 pattern ./extensions/CirrusSearch/tests/selenium/specs/**/*.js did not match any file
00:01:19.441 pattern /home/jenkins/workspace/selenium-Popups-jessie/src/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
00:01:19.441 pattern /home/jenkins/workspace/selenium-Popups-jessie/src/skins/*/tests/selenium/specs/**/*.js did not match any file
...
00:01:30.301 [chrome #0-0] Spec: /home/jenkins/workspace/selenium-Popups-jessie/src/extensions/Popups/tests/selenium/specs/previews.js
00:01:30.301 [chrome #0-0] Running: chrome
00:01:30.301 [chrome #0-0]
00:01:30.301 [chrome #0-0]   Dwelling on a valid link
00:01:30.302 [chrome #0-0]
00:01:30.302 [chrome #0-0]   Dwelling on a valid link
00:01:30.302 [chrome #0-0]       ✓ I should see a preview
00:01:30.302 [chrome #0-0]       ✓ Abandoning link hides preview
00:01:30.303 [chrome #0-0]
00:01:30.303 [chrome #0-0]
00:01:30.303 [chrome #0-0] 2 passing (11s)
...
00:01:30.335 2 passing (11.60s)
zeljkofilipin updated the task description. (Show Details)Apr 27 2018, 1:59 PM
zeljkofilipin added a comment.Apr 27 2018, 2:05 PM

selenium-RelatedArticles-jessie/313/console

00:01:16.853 + cd /home/jenkins/workspace/selenium-RelatedArticles-jessie/src
...
00:01:16.854 + ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/RelatedArticles/tests/selenium/specs/readmore.js
...
00:01:17.754 pattern /home/jenkins/workspace/selenium-RelatedArticles-jessie/src/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
00:01:17.754 pattern /home/jenkins/workspace/selenium-RelatedArticles-jessie/src/skins/*/tests/selenium/specs/**/*.js did not match any file
00:01:17.934 pattern ./extensions/CirrusSearch/tests/selenium/specs/**/*.js did not match any file
00:01:18.648 pattern /home/jenkins/workspace/selenium-RelatedArticles-jessie/src/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
00:01:18.649 pattern /home/jenkins/workspace/selenium-RelatedArticles-jessie/src/skins/*/tests/selenium/specs/**/*.js did not match any file
...
00:01:24.718 [chrome #0-0] Spec: /home/jenkins/workspace/selenium-RelatedArticles-jessie/src/extensions/RelatedArticles/tests/selenium/specs/readmore.js
00:01:24.719 [chrome #0-0] Running: chrome
00:01:24.719 [chrome #0-0]
00:01:24.719 [chrome #0-0]   ReadMore
00:01:24.719 [chrome #0-0]
00:01:24.719 [chrome #0-0]   ReadMore
00:01:24.719 [chrome #0-0]       ✓ ReadMore is not present on Vector
00:01:24.720 [chrome #0-0]       ✓ ReadMore is present in Minerva
00:01:24.720 [chrome #0-0]
00:01:24.720 [chrome #0-0]
00:01:24.720 [chrome #0-0] 2 passing (6s)
...
00:01:24.750 2 passing (7.00s)
zeljkofilipin added a comment.Apr 27 2018, 2:43 PM

The problem is reproducible outside of CI, it fails on my machine too.

~/Documents/gerrit/mediawiki/vagrant/mediawiki$ export MEDIAWIKI_PASSWORD=not-the-real-one
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ export MEDIAWIKI_USER='Selenium user'
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ export MW_SERVER=https://en.wikipedia.beta.wmflabs.org:443
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ export MW_SCRIPT_PATH=/w
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/Popups/tests/selenium/specs/previews.js
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
...
[chrome #0-0] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/Popups/tests/selenium/specs/previews.js
[chrome #0-0] Running: chrome
[chrome #0-0]
[chrome #0-0]   Dwelling on a valid link
[chrome #0-0]
[chrome #0-0]   Dwelling on a valid link
[chrome #0-0]       ✓ I should see a preview
[chrome #0-0]       ✓ Abandoning link hides preview
[chrome #0-0]
[chrome #0-0]
[chrome #0-0] 2 passing (12s)
[chrome #0-0]

Wrote xunit report to [./log/].

2 passing (12.30s)
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/RelatedArticles/tests/selenium/specs/readmore.js 
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
[16:38:57] [S] [MWBOT] Login successful: Selenium user@https://en.wikipedia.beta.wmflabs.org:443/w
------------------------------------------------------------------
[chrome #0-0] Session ID: a5bbf8c7789e2ce77ac0e4bd5b9f1a28
[chrome #0-0] Spec: /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/RelatedArticles/tests/selenium/specs/readmore.js
[chrome #0-0] Running: chrome
[chrome #0-0]
[chrome #0-0]   ReadMore
[chrome #0-0]
[chrome #0-0]   ReadMore
[chrome #0-0]       ✓ ReadMore is not present on Vector
[chrome #0-0]       ✓ ReadMore is present in Minerva
[chrome #0-0]
[chrome #0-0]
[chrome #0-0] 2 passing (10s)
[chrome #0-0]

Wrote xunit report to [./log/].

2 passing (11.00s)
~/Documents/gerrit/mediawiki/vagrant/mediawiki$ ./node_modules/.bin/wdio tests/selenium/wdio.conf.js --spec extensions/CirrusSearch/tests/selenium/specs/smoke_test.js 
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
zeljkofilipin added a comment.EditedApr 27 2018, 2:46 PM

It's because of this in wdio.conf.js introduced in 424763.

 // Patterns to exclude.
 exclude: [
   './extensions/CirrusSearch/tests/selenium/specs/**/*.js'
],
zeljkofilipin added a comment.EditedApr 27 2018, 3:18 PM

A simple solution would be to overwrite exclude with and empty array (see 429435):

wdio.smoke.js

const merge = require( 'deepmerge' ),
	wdioConf = require( './wdio.conf.js' );

// Overwrite default settings
exports.config = merge( wdioConf.config, {
	exclude: []
} );

but looks like it does not work, exclude is still [ './extensions/CirrusSearch/tests/selenium/specs/**/*.js' ]

~/Documents/gerrit/mediawiki/vagrant/mediawiki$ ./node_modules/.bin/wdio tests/selenium/wdio.smoke.js --spec extensions/CirrusSearch/tests/selenium/specs/smoke_test.js 
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js did not match any file
pattern /Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/skins/*/tests/selenium/specs/**/*.js did not match any file
[ { maxInstances: 1,
    browserName: 'chrome',
    chromeOptions: { args: [Array] } } ]
{ host: '127.0.0.1',
  port: 4444,
  sync: true,
  specs: 
   [ '/Users/z/Documents/gerrit/mediawiki/vagrant/mediawiki/extensions/CirrusSearch/tests/selenium/specs/smoke_test.js' ],
  suites: {},
  exclude: [ './extensions/CirrusSearch/tests/selenium/specs/**/*.js' ],
  logLevel: 'error',
  coloredLogs: true,
  deprecationWarnings: true,
  baseUrl: 'https://en.wikipedia.beta.wmflabs.org:443/w',
  bail: 0,
  waitforInterval: 500,
  waitforTimeout: 10000,
  framework: 'mocha',
  reporters: [ 'spec', 'junit' ],
  reporterOptions: { junit: { outputDir: './log/' } },
  maxInstances: 1,
  maxInstancesPerCapability: 100,
  connectionRetryTimeout: 90000,
  connectionRetryCount: 3,
  debug: false,
  execArgv: null,
  mochaOpts: { timeout: 20000, ui: 'bdd' },
  jasmineNodeOpts: { defaultTimeoutInterval: 10000 },
  onPrepare: [Function: NOOP],
  before: [],
  beforeSession: [],
  beforeSuite: [],
  beforeHook: [],
  beforeTest: [],
  beforeCommand: [],
  afterCommand: [],
  afterTest: [ [Function: bound afterTest] ],
  afterHook: [],
  afterSuite: [],
  afterSession: [],
  after: [],
  onComplete: [Function: onComplete],
  onError: [],
  onReload: [],
  beforeFeature: [],
  beforeScenario: [],
  beforeStep: [],
  afterFeature: [],
  afterScenario: [],
  afterStep: [],
  username: 'Selenium user',
  password: 'tnV9YtF5dpUUhd8',
  services: [ 'sauce' ],
  user: undefined,
  key: undefined,
  screenshotPath: './log/',
  spec: 'extensions/CirrusSearch/tests/selenium/specs/smoke_test.js',
  watch: false }
gerritbot subscribed.Apr 27 2018, 3:20 PM

Change 429435 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP selenium-CirrusSearch-jessie does not run any tests

https://gerrit.wikimedia.org/r/429435

gerritbot added a project: Patch-For-Review.Apr 27 2018, 3:20 PM
zeljkofilipin added a subscriber: hashar.EditedApr 27 2018, 3:29 PM

A simple example seems to work fine (thanks @hashar):

// $ node

> const merge = require('deepmerge');
undefined

> let config = {a:1, b:2};
undefined

> config
{ a: 1, b: 2 }

> let configChange = {b:3};
undefined

> configChange
{ b: 3 }

> merge(config, configChange);
{ a: 1, b: 3 }
• debt moved this task from needs triage to watching / waiting on the Discovery-Search board.May 3 2018, 5:29 PM
zeljkofilipin added a comment.EditedMay 4 2018, 4:21 PM
// $ node

> const merge = require('deepmerge');
undefined

> let config = {a:[1]};
undefined

> config
{ a: [ 1 ] }

> let configChange = {a:[2]};
undefined

> configChange
{ a: [ 2 ] }

> merge(config, configChange);
{ a: [ 1, 2 ] }

configChange does not override config, just adds to it.

zeljkofilipin added a comment.May 4 2018, 4:51 PM

From KyleAMathews/deepmerge#arraymerge

// $ node

> const merge = require('deepmerge');
undefined

> let config = {a:[1]};
undefined

> config
{ a: [ 1 ] }

> let configChange = {a:[2]};
undefined

> configChange
{ a: [ 2 ] }

> merge(config, configChange);
{ a: [ 1, 2 ] }

> function overwriteMerge(destinationArray, sourceArray, options) {
... 	return sourceArray
... }
undefined

> merge(config, configChange,{ arrayMerge: overwriteMerge });
{ a: [ 2 ] }
zeljkofilipin added a comment.EditedMay 4 2018, 5:16 PM

Created test job: selenium-CirrusSearch-jessie-429435

...
00:01:21.339 [chrome #0-0] Spec: /home/jenkins/workspace/selenium-CirrusSearch-jessie-429435/src/extensions/CirrusSearch/tests/selenium/specs/smoke_test.js
00:01:21.339 [chrome #0-0] Running: chrome
00:01:21.339 [chrome #0-0]
00:01:21.339 [chrome #0-0]   Smoke test for search
00:01:21.339 [chrome #0-0]
00:01:21.339 [chrome #0-0]   Smoke test for search
00:01:21.339 [chrome #0-0]       ✓ Search suggestions
00:01:21.340 [chrome #0-0]       ✓ Fill in search term and click search
00:01:21.340 [chrome #0-0]       ✓ Search with accent yields result page with accent
00:01:21.340 [chrome #0-0]
00:01:21.340 [chrome #0-0]
00:01:21.340 [chrome #0-0] 3 passing (8s)
...
zeljkofilipin added a comment.EditedMay 4 2018, 5:42 PM

Minor job change is needed, see 430944 and selenium-CirrusSearch-jessie-429435/jobConfigHistory.

gerritbot added a comment.May 4 2018, 5:44 PM

Change 430944 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] selenium: Do not exclude CirrusSearch when running smoke tests

https://gerrit.wikimedia.org/r/430944

zeljkofilipin added subscribers: EBernhardson, dcausse.May 10 2018, 3:51 PM
zeljkofilipin moved this task from Deep work 🌊 to Quarter 👔 on the User-zeljkofilipin board.May 18 2018, 1:09 PM
zeljkofilipin moved this task from Quarter 👔 to Deep work 🌊 on the User-zeljkofilipin board.May 24 2018, 9:36 AM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Jun 4 2018, 2:49 PM
zeljkofilipin moved this task from Deep work 🌊 to Quarter 👔 on the User-zeljkofilipin board.
zeljkofilipin moved this task from In-progress to Backlog on the Release-Engineering-Team (Kanban) board.Jun 4 2018, 4:33 PM
zeljkofilipin mentioned this in T198389: Q1 Selenium framework improvements.Jun 28 2018, 4:28 PM
zeljkofilipin edited parent tasks, added: T198389: Q1 Selenium framework improvements; removed: T190994: Q4 Selenium framework improvements.
zeljkofilipin moved this task from Quarter 👔 to Backlog 🪒 on the User-zeljkofilipin board.Jun 28 2018, 5:05 PM
• Vvjjkkii renamed this task from selenium-CirrusSearch-jessie does not run any tests to f3daaaaaaa.Jul 1 2018, 1:13 AM
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail, TCB-Team (now WMDE-TechWish).
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from f3daaaaaaa to selenium-CirrusSearch-jessie does not run any tests.Jul 2 2018, 2:00 PM
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot removed projects: TCB-Team (now WMDE-TechWish), Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
CommunityTechBot added subscribers: gerritbot, Aklapper.
zeljkofilipin mentioned this in T188742: Run tests daily targeting beta cluster for all repositories with Selenium tests.Jul 5 2018, 11:24 AM
gerritbot added a comment.Jul 9 2018, 3:51 PM

Change 444609 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/CirrusSearch@master] Selenium: create selenium-daily NPM script

https://gerrit.wikimedia.org/r/444609

zeljkofilipin updated the task description. (Show Details)Jul 9 2018, 3:52 PM
gerritbot added a comment.Jul 12 2018, 7:34 PM

Change 444609 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Selenium: Create selenium-daily NPM script

https://gerrit.wikimedia.org/r/444609

zeljkofilipin claimed this task.Jul 13 2018, 2:57 PM
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.
zeljkofilipin moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin closed this task as Resolved.Jul 13 2018, 2:59 PM

Resolved by CirrusSearch-npm-browser-run-selenium-daily-node-6-docker.

gerritbot added a comment.Jul 13 2018, 3:18 PM

Change 429435 abandoned by Zfilipin:
selenium: Do not exclude CirrusSearch when running smoke tests

Reason:
Resolved by T188742

https://gerrit.wikimedia.org/r/429435

gerritbot added a comment.Jul 13 2018, 3:23 PM

Change 430944 abandoned by Zfilipin:
selenium: Do not exclude CirrusSearch when running smoke tests

Reason:
Resolved by T188742

https://gerrit.wikimedia.org/r/430944

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.Feb 12 2025, 9:35 AM
Maintenance_bot removed a project: Patch-For-Review.Feb 12 2025, 10:31 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits