Page MenuHomePhabricator

selenium-daily-beta-Wikibase Jenkins job fails with: AssertionError: 'Q1xy' === 'Item:Q1xy'
Closed, ResolvedPublic5 Story Points

Description

Report:

selenium-daily-beta-Wikibase Jenkins job fails:

> MEDIAWIKI_USER='Selenium user' MW_SERVER=https://wikidata.beta.wmflabs.org:443 MW_SCRIPT_PATH=/w npm run selenium-test


> Wikibase@0.0.0 selenium-test /src
> wdio repo/tests/selenium/wdio.conf.js

[46.366][SEVERE]: Failed to create /nonexistent/.pki/nssdb directory.

	Screenshot: /log/the-title-should-match.png

------------------------------------------------------------------
[chrome #0-0] Session ID: 456eac05a76dc7e4d1fdb7df4eb1fffc
[chrome #0-0] Spec: /src/repo/tests/selenium/specs/nonexisting.item.js
[chrome #0-0] Running: chrome
[chrome #0-0]
[chrome #0-0] WikibaseRepoNonExistingItemPage
[chrome #0-0]   ✓ edit tab does should not be there
[chrome #0-0]   1) the title should match
[chrome #0-0]
[chrome #0-0]
[chrome #0-0] 1 passing (7s)
[chrome #0-0] 1 failing
[chrome #0-0]
[chrome #0-0] 1) WikibaseRepoNonExistingItemPage the title should match:
[chrome #0-0] 'Q1xy' === 'Item:Q1xy'
[chrome #0-0] AssertionError: 'Q1xy' === 'Item:Q1xy'
[chrome #0-0]     at Context.<anonymous> (/src/repo/tests/selenium/specs/nonexisting.item.js:15:10)
[chrome #0-0]     at Promise.F (/src/node_modules/core-js/library/modules/_export.js:36:28)

Screenshot from the job:

Diagnosis:

@Addshore said:

This is down to the configuration / hardcoding of the namespace.

The test does this:

NonExistingItemPage.open();
// Which does the following internally page.openTitle( 'Item:Q1xy' );

assert.strictEqual( NonExistingItemPage.title.getText(), 'Item:Q1xy' );

beta is setup in the same was as wikidata.org which does not use the Item: namespace for items, instead it uses the mainspace.
We probably need some way to detect or pass in the item namespace of the wiki that the test is running on?

Acceptance Criteria:

  • The test does not have a hard coded namespace but instead gets the namespace from somewhere
  • Test pases on beta

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 13 2018, 11:09 AM

@Ladsgroup @Addshore do you know what is the problem?

This is down to the configuration / hardcoding of the namespace.

The test does this:

NonExistingItemPage.open();
// Which does the following internally page.openTitle( 'Item:Q1xy' );

assert.strictEqual( NonExistingItemPage.title.getText(), 'Item:Q1xy' );

beta is setup in the same was as wikidata.org which does not use the Item: namespace for items, instead it uses the mainspace.
We probably need some way to detect or pass in the item namespace of the wiki that the test is running on?

As far as I am concerned, it's fine not to run a failing test, until it's fixed. See how it's done for core: https://gerrit.wikimedia.org/r/c/mediawiki/core/+/457854

Addshore triaged this task as Normal priority.Oct 8 2018, 10:56 AM
Addshore raised the priority of this task from Normal to Needs Triage.
Addshore triaged this task as Normal priority.Nov 6 2018, 1:56 PM
Addshore added a project: Browser-Tests.
Addshore moved this task from Needs Work to Ready to estimate on the Wikidata-Campsite board.
Addshore updated the task description. (Show Details)Nov 6 2018, 1:58 PM
Addshore updated the task description. (Show Details)Nov 6 2018, 2:18 PM
Addshore set the point value for this task to 5.

Possible solutions considered:

  • Get settings from the API? (There isn't an obvious API)
  • Pass in an ENV var stating what the namespace is? (similar to the ruby ones)
  • try to get the namespace from the JS via the browser in the tests
  • Be more relaxed about how we check / find the page

Task breakdown proposed solution:

Change 472181 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/extensions/Wikibase@master] Make nonexisting item test agnostic of item namespace

https://gerrit.wikimedia.org/r/472181

Change 472181 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Make nonexisting item test agnostic of item namespace

https://gerrit.wikimedia.org/r/472181

Addshore closed this task as Resolved.Nov 8 2018, 4:18 PM

Verified on the daily beta jenkins job i just ran:

https://integration.wikimedia.org/ci/job/selenium-daily-beta-Wikibase/77/console

00:01:26.313 ------------------------------------------------------------------
00:01:26.313 [chrome #0-1] Session ID: 1baed21bb2ef59f5cb2d471bb6664506
00:01:26.313 [chrome #0-1] Spec: /src/repo/tests/selenium/specs/nonexisting.item.js
00:01:26.314 [chrome #0-1] Running: chrome
00:01:26.314 [chrome #0-1]
00:01:26.314 [chrome #0-1] WikibaseRepoNonExistingItemPage
00:01:26.314 [chrome #0-1]   ✓ edit tab does should not be there
00:01:26.315 [chrome #0-1]   ✓ the title should match
00:01:26.315 [chrome #0-1]
00:01:26.315 [chrome #0-1]
00:01:26.315 [chrome #0-1] 2 passing (3s)
00:01:26.316 [chrome #0-1]