Page MenuHomePhabricator

UnderflowException when creating an account, FancyCaptcha.php: Ran out of captcha images
Closed, ResolvedPublic

Description

This occurs when registering a new account on english beta wikipedia.

[W-WBqKwQBGoAADimdgUAAAAQ] /w/index.php?title=Special:CreateAccount&returnto=Main+Page UnderflowException from line 446 of /srv/mediawiki/php-master/extensions/ConfirmEdit/FancyCaptcha/includes/FancyCaptcha.php: Ran out of captcha images

Backtrace:

#0 /srv/mediawiki/php-master/extensions/ConfirmEdit/SimpleCaptcha/SimpleCaptcha.php(1126): FancyCaptcha->getCaptcha()
#1 /srv/mediawiki/php-master/extensions/ConfirmEdit/includes/auth/CaptchaPreAuthenticationProvider.php(60): SimpleCaptcha->createAuthenticationRequest()
#2 /srv/mediawiki/php-master/includes/auth/AuthManager.php(2095): CaptchaPreAuthenticationProvider->getAuthenticationRequests(string, array)
#3 /srv/mediawiki/php-master/includes/auth/AuthManager.php(2073): MediaWiki\Auth\AuthManager->getAuthenticationRequestsInternal(string, array, array, User)
#4 /srv/mediawiki/php-master/includes/specialpage/AuthManagerSpecialPage.php(256): MediaWiki\Auth\AuthManager->getAuthenticationRequests(string, User)
#5 /srv/mediawiki/php-master/includes/specialpage/LoginSignupSpecialPage.php(141): AuthManagerSpecialPage->loadAuth(NULL)
#6 /srv/mediawiki/php-master/includes/specialpage/LoginSignupSpecialPage.php(229): LoginSignupSpecialPage->load(NULL)
#7 /srv/mediawiki/php-master/includes/specialpage/SpecialPage.php(569): LoginSignupSpecialPage->execute(NULL)
#8 /srv/mediawiki/php-master/includes/specialpage/SpecialPageFactory.php(558): SpecialPage->run(NULL)
#9 /srv/mediawiki/php-master/includes/MediaWiki.php(288): MediaWiki\Special\SpecialPageFactory->executePath(Title, RequestContext)
#10 /srv/mediawiki/php-master/includes/MediaWiki.php(862): MediaWiki->performRequest()
#11 /srv/mediawiki/php-master/includes/MediaWiki.php(517): MediaWiki->main()
#12 /srv/mediawiki/php-master/index.php(42): MediaWiki->run()
#13 /srv/mediawiki/w/index.php(3): include(string)
#14 {main}

Event Timeline

krenair@deployment-deploy01:~$ ./deployment-tin-home/get-words-lists.sh 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   141  100   141    0     0    293      0 --:--:-- --:--:-- --:--:--   293
100 4135k  100 4135k    0     0  4698k      0 --:--:-- --:--:-- --:--:-- 4698k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   171  100   171    0     0   2177      0 --:--:-- --:--:-- --:--:--  2192
100  3566  100  3566    0     0  26941      0 --:--:-- --:--:-- --:--:-- 26941
krenair@deployment-deploy01:~$ sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php extensions/ConfirmEdit/maintenance/GenerateFancyCaptchas.php aawiki --wordlist=/tmp/words --font=/usr/share/fonts/truetype/freefont/FreeMonoBoldOblique.ttf --blacklist=/tmp/badwords --fill=1 --verbose --delete
Generating 1 new captchas.. Done.

Generated 1 captchas in 0.4 seconds
Getting a list of old captchas to delete... Done.
Copying the new captchas to storage...Errored.
An unknown error occurred in storage backend "global-swift-eqiad".

Removing temporary files... Done.

Whole captchas generation process took 0.6 seconds

can't seem to generate any number of captchas, they all fail copying to swift

krenair@deployment-ms-be03:~$ sudo swift-recon --human-readable --diskusage
===============================================================================
--> Starting reconnaissance on 2 hosts (object)
===============================================================================
[2018-11-21 22:30:43] Checking disk usage now
-> http://10.68.22.125:6000/recon/diskusage: <urlopen error [Errno 113] EHOSTUNREACH>
-> http://10.68.16.139:6000/recon/diskusage: <urlopen error [Errno 113] EHOSTUNREACH>
No hosts returned valid data.
===============================================================================

After running what must have been over a hundred different swift-ring-builder commands across -ms-be03, -ms-be04, and -ms-fe02, I think Swift is now working: https://upload.beta.wmflabs.org/wikipedia/en/5/5d/Screen_Shot_2018-10-29_at_2.24.02_PM.png

Captcha generation process now running.

So while that was having the same old problems, puppet reversed all my swift changes. Didn't realise that was puppetised.

Hopefully Puppet won't do that again now. Running captcha generation again.

krenair@deployment-deploy01:~$ sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php extensions/ConfirmEdit/maintenance/GenerateFancyCaptchas.php aawiki --wordlist=/tmp/words --font=/usr/share/fonts/truetype/freefont/FreeMonoBoldOblique.ttf --blacklist=/tmp/badwords --fill=900 --verbose --delete
Generating 900 new captchas.. Done.

Generated 900 captchas in 63.8 seconds
Getting a list of old captchas to delete... Done.
Copying the new captchas to storage... Done.

Copied 900 captchas to storage in 11.2 seconds
Deleting 953 old captchas...
Done.

Deleted 953 old captchas in 7.5 seconds
Removing temporary files... Done.

Whole captchas generation process took 83.1 seconds

But when I go to test it I find the database is read-only for some reason. That doesn't make sense - replication between the database hosts appears okay.

This problem is gone now, some work here was part of T208101: Migrate deployment-prep to eqiad1

@Krenair thank you for sorting this out quickly!