Page MenuHomePhabricator

GettingStarted failing on Jenkins due to CAPTCHA bypass
Closed, ResolvedPublic2 Story Points

Description

GettingStarted is failing on Jenkins. @phuedx looked at it, and the CAPTCHA bypass is failing. See T90451: Configure Beta Cluster to bypass the CAPTCHA when testing GettingStarted user registration.

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptJan 20 2017, 1:02 AM
phuedx added a comment.Feb 8 2017, 1:16 PM

@zeljkofilipin: How can I see which environment variables are being set during a build? I'd like to verify that mediawiki_captcha_bypass_password is set and correct.

Build #268 is the last one that is green:

Success Build #268 (Jan 11, 2017 12:22:59 PM)

Failures have started with #269:

Failed Build #269 (Jan 12, 2017 12:22:59 PM)

Job configuration history is unfortunately empty :( so we can not see if anything changed there:

https://integration.wikimedia.org/ci/view/Selenium/job/selenium-GettingStarted/jobConfigHistory/

zeljkofilipin added a subscriber: hashar.EditedFeb 9 2017, 10:01 AM

@zeljkofilipin: How can I see which environment variables are being set during a build?

I see I did not answer the question. I am not sure if there is a list of environment variables per job or build. Maybe @hashar would know.

I'd like to verify that mediawiki_captcha_bypass_password is set and correct.

This is doable. You can edit and existing job, or even better, copy a job, and add echo $MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD to the shell script that runs the tests. Run the job, copy/paste the output and delete the build and/or job immediately.

There is a small security risk, since console output will have the password. The risk can be minimized by deleting the build/job immediately after it finishes. (Our Jenkins installation is wide open. There might be a way to make a build/job private.)

There are some instructions on how to copy a job. You need to do just a few of the first steps:

https://www.mediawiki.org/wiki/Reading/Web/QA#Simulating_browser_test_run_on_an_unmerged_patchset.

I have added echo $MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD just above rake selenium in a copy of selenium-GettingStarted job:

echo $MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD

# run the tests
"$WORKSPACE"/vendor/bin/bundle exec rake selenium

The output showed that indeed the password is not the same as at the office wiki. I have updated the password in Jenkins credential store (pasted the value from office wiki) and restarted the selenium-GettingStarted job. The job still fails.

@phuedx I think MEDIAWIKI_CAPTCHA_BYPASS_PASSWORD is not correct. Could you please make sure both office wiki and Jenkins have the correct password?

Change 336770 had a related patch set uploaded (by Phuedx):
Fill required field when bypassing CAPTCHA

https://gerrit.wikimedia.org/r/336770

In theory if one want to decrypt a credential password:

Head to contint1001 and find the encrypted credential in /var/lib/jenkins/credentials.xml

Head to Jenkins Groovy console https://integration.wikimedia.org/ci/script and fill in:

println(hudson.util.Secret.decrypt("CRYPTIC SECRET HERE"))

Press Run and that should gives you the credential.

Change 336770 merged by jenkins-bot:
Fill required field when bypassing CAPTCHA

https://gerrit.wikimedia.org/r/336770

zeljkofilipin closed this task as Resolved.Feb 9 2017, 10:34 AM
zeljkofilipin claimed this task.

Resolved by @phuedx in https://gerrit.wikimedia.org/r/336770. The job is back to green.

phuedx added a subscriber: greg.Feb 9 2017, 10:36 AM

Note that while trying to figure this out, I did update the mediawiki_captcha_bypass_password credential in Jenkins to match the value of $wmgCaptchaPassword in /srv/mediawiki-staging/private/PrivateSettings.php on tin, which I thought was the correct deployment server. It didn't make the build pass so obviously I was wrong. I reset the credential immediately after the build failed.

Why would PrivateSettings.php be out of sync?

phuedx added a project: Unplanned-Sprint-Work.
phuedx set the point value for this task to 2.

Note that while trying to figure this out, I did update the mediawiki_captcha_bypass_password credential in Jenkins to match the value of $wmgCaptchaPassword in /srv/mediawiki-staging/private/PrivateSettings.php on tin, which I thought was the correct deployment server. It didn't make the build pass so obviously I was wrong. I reset the credential immediately after the build failed.
Why would PrivateSettings.php be out of sync?

Do you mean actual tin, or deployment-tin?

This runs on Beta Cluster, so it would be deployment-tin. tin is production.

Do you mean actual tin, or deployment-tin?
This runs on Beta Cluster, so it would be deployment-tin. tin is production.

It's highly likely that I ssh'd to the wrong server.