Page MenuHomePhabricator

Browser Test Failures with Password Blacklist patches
Closed, ResolvedPublic

Description

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/414603/

wmf-quibble-core-vendor-mysql-hhvm-docker FAILURE in 13m 07s
mediawiki-quibble-composer-mysql-php70-docker FAILURE in 5m 54s
mediawiki-quibble-vendor-mysql-hhvm-docker FAILURE in 4m 20s
mediawiki-quibble-vendor-mysql-php70-docker FAILURE in 7m 07s

Possibly related to password requirement changes... The list of passwords that can't be used is https://github.com/wikimedia/password-blacklist/blob/master/scripts/data/10_million_password_list_top_100000.txt - Chances are if it's something simple/common it's probably going to be a problem

Event Timeline

Reedy created this task.Sep 17 2018, 5:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Reedy updated the task description. (Show Details)Sep 17 2018, 5:37 PM
Reedy added a comment.Sep 17 2018, 5:51 PM

It looks like we need to change the passwords of selenium-user-beta and selenium-user-production (proactively) under https://integration.wikimedia.org/ci/credentials/store/system/domain/selenium/

It looks like we need to change the passwords of selenium-user-beta and selenium-user-production (proactively) under https://integration.wikimedia.org/ci/credentials/store/system/domain/selenium/

No. Those are only for Jenkins jobs that run daily and target beta/production, selenium-daily-beta-PROJECT, selenium-PROJECT, selenium-PROJECT-chrome. Quibble jobs don't use beta/production password.

zeljkofilipin triaged this task as Low priority.Sep 18 2018, 12:54 PM
zeljkofilipin added a subscriber: hashar.

I'm looking where the password for Quibble jobs is stored. Maybe @hashar would know.

The last time password was updated in integration/jenkins/415312.

Change 461138 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/libs/PasswordBlacklist@master] Add testwikijenkinspass to passwords to test not blacklisted

https://gerrit.wikimedia.org/r/461138

This is currently in bin/mw-set-env.sh

# MediaWiki admin user
# Used by mw-install* scripts for creating the account,
# but also by used by selenium tests for logging-in
export MEDIAWIKI_USER="WikiAdmin"
export MEDIAWIKI_PASSWORD="testwikijenkinspass"
zeljkofilipin added a subscriber: zeljkofilipin.

Assigning to @Reedy since he already created the commit that should fix the problem.

Reedy added a comment.Sep 18 2018, 2:32 PM

My patch isn't going to fix it, just testing whether the blacklist is blacklisting that password... I've added it to the core test too for completeness

It looks like we need to change the passwords of selenium-user-beta and selenium-user-production (proactively) under https://integration.wikimedia.org/ci/credentials/store/system/domain/selenium/

So this was a guess of what was happening... Unfortunately the error messages are vague as to telling me why anything is failing

Reedy added a comment.Sep 18 2018, 5:38 PM

So... Error: Unrecognised hook ["after each" hook for "should be deletable"] for suite [Page] ???

14:49:07 [14:49:07] [E] [MWBOT] Login failed: WikiAdmin@http://127.0.0.1:9412/
14:49:08 
14:49:08 	Screenshot: /workspace/log/should-be-re-creatable.png
14:49:08 
14:49:08 
14:49:08 	Screenshot: /workspace/log/should-be-re-creatable.png
14:49:08 
14:49:10 [14:49:10] [E] [MWBOT] Login failed: WikiAdmin@http://127.0.0.1:9412/
14:49:10 [14:49:10] [E] [MWBOT] Login failed: WikiAdmin@http://127.0.0.1:9412/
14:49:10 (node:2517) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 4): TypeError: Cannot read property 'currentRetry' of undefined
14:49:11 
14:49:11 	Screenshot: /workspace/log/should-have-history-%40daily.png
14:49:11 
14:49:11 
14:49:11 	Screenshot: /workspace/log/should-have-history-%40daily.png
14:49:11 
14:49:11 /workspace/src/node_modules/webdriverio/build/lib/utils/ReporterStats.js:362
14:49:11             if (!suiteStats.hooks[uid]) throw Error(`Unrecognised hook [${runner.title}] for suite [${runner.parent}]`);
14:49:11                                         ^
14:49:11 
14:49:11 Error: Unrecognised hook ["after each" hook for "should be deletable"] for suite [Page]
14:49:11     at Error (native)
14:49:11     at ReporterStats.getHookStats (/workspace/src/node_modules/webdriverio/build/lib/utils/ReporterStats.js:362:47)
14:49:11     at ReporterStats.hookEnd (/workspace/src/node_modules/webdriverio/build/lib/utils/ReporterStats.js:332:34)
14:49:11     at BaseReporter.<anonymous> (/workspace/src/node_modules/webdriverio/build/lib/utils/BaseReporter.js:147:25)
14:49:11     at emitOne (events.js:96:13)
14:49:11     at BaseReporter.emit (events.js:188:7)
14:49:11     at BaseReporter.handleEvent (/workspace/src/node_modules/webdriverio/build/lib/utils/BaseReporter.js:300:27)
14:49:11     at Launcher.messageHandler (/workspace/src/node_modules/webdriverio/build/lib/launcher.js:688:28)
14:49:11     at emitTwo (events.js:106:13)
14:49:11     at ChildProcess.emit (events.js:191:7)
14:49:11
Reedy added a comment.EditedSep 18 2018, 6:47 PM

Ok... So

https://gerrit.wikimedia.org/r/#/c/mediawiki/libs/PasswordBlacklist/+/461138/ confirms the password blacklist isn't rejecting testwikijenkinspass as is supposed to be used based on https://gerrit.wikimedia.org/r/#/c/integration/jenkins/+/415312/5/bin/mw-set-env.sh

https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/414603/ adds the same password to core tests, and that doesn't fail either...

On top of the core patch, I've made another to core, disabling the password blacklisting checks for all the groups - https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/461175/ the tests pass. Narrowed to just for sysops, and that's causing the failure

So either it's causing an unintended browser test failure (for some reason), or the password being used by the failing test isn't the one we think it is for the sysop user (ie it's not actually testwikijenkinspass, it's some other blacklisted password)

18:38:53 [chrome #0-1] Could not login: Aborted

This error is kinda useless all around

Then there's errors like...

18:38:53 (node:1451) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 4): TypeError: Cannot read property 'isPending' of undefined

Reedy removed Reedy as the assignee of this task.Sep 18 2018, 6:47 PM
wmf-quibble-core-vendor-mysql-hhvm-docker FAILURE in 13m 07s
mediawiki-quibble-composer-mysql-php70-docker FAILURE in 5m 54s
mediawiki-quibble-vendor-mysql-hhvm-docker FAILURE in 4m 20s
mediawiki-quibble-vendor-mysql-php70-docker FAILURE in 7m 07s

integration/jenkins.git and bin/mw-set-env.sh were used with the legacy jobs running on Nodepool. They are deprecated and not much used anymore. Instead the logic to install MediaWiki and run tests has been ported to Quibble, it uses the dummy password testpass.

https://github.com/wikimedia/integration-quibble/blob/3da498c/quibble/mediawiki/maintenance.py#L47

quibble/mediawiki/maintenance.py
def install(args, mwdir=None):
    log = logging.getLogger('mw.maintenance.install')

    cmd = ['php', 'maintenance/install.php']
    cmd.extend(args)
    cmd.extend([
        '--with-extensions',  # T189567
        '--pass=testpass',
        'TestWiki',
        'WikiAdmin'
    ])
    log.info(' '.join(cmd))

The Selenium tests are also being launched by Quibble, the password is set via the environment variable MEDIAWIKI_PASSWORD:

quibble/test.py
def run_webdriver(mwdir, display, port=9412):
    webdriver_env = {}
    webdriver_env.update(os.environ)
    webdriver_env.update({
        'MW_SERVER': 'http://127.0.0.1:%s' % port,
        'MW_SCRIPT_PATH': '/',
        'FORCE_COLOR': '1',  # for 'supports-color'
        'MEDIAWIKI_USER': 'WikiAdmin',      #  /
        'MEDIAWIKI_PASSWORD': 'testpass',   # <------------- here! -----------------
        'DISPLAY': display,                 #  \
    })

So I guess we can change the password in Quibble and that will solve it. Need a patch for integration/quibble, test it is actually working, then we cut a version, update the Docker containers and refresh all Jenkins jobs. Should be testable with:

$ ZUUL_URL=https://gerrit.wikimedia.org/r/p ZUUL_PROJECT=mediawiki/core ZUUL_REF=refs/changes/414603/24 quibble --run=selenium

Change 461195 had a related patch set uploaded (by Reedy; owner: Reedy):
[integration/quibble@master] Use stronger password in quibble related browser tests

https://gerrit.wikimedia.org/r/461195

Reedy added a comment.Sep 18 2018, 7:29 PM

Thanks @hashar!

Patch above to update the passwords there... :)

Change 461138 merged by jenkins-bot:
[mediawiki/libs/PasswordBlacklist@master] Add testwikijenkinspass to passwords to test not blacklisted

https://gerrit.wikimedia.org/r/461138

greg assigned this task to hashar.Oct 3 2018, 7:00 PM
greg added a subscriber: greg.

How do we actually test this then?

I think @hashar needs to make a new quibble release.

Change 465606 had a related patch set uploaded (by Reedy; owner: Reedy):
[integration/config@master] Rebuild quibble to bring in I79c015b2a8a029eeb8746c1fad201b6e801935b3

https://gerrit.wikimedia.org/r/465606

Change 461195 merged by jenkins-bot:
[integration/quibble@master] Use stronger password in quibble related browser tests

https://gerrit.wikimedia.org/r/461195

Change 465606 merged by jenkins-bot:
[integration/config@master] Bump Quibble images to 0.0.27

https://gerrit.wikimedia.org/r/465606

Reedy closed this task as Resolved.Oct 13 2018, 11:20 AM
Reedy removed a project: Patch-For-Review.

Thanks!

Reedy reassigned this task from hashar to Legoktm.Oct 13 2018, 11:20 AM