Page MenuHomePhabricator
Create Task
Maniphest T107498

Redis error on Flow maintenance script on testwiki on Beta Cluster
Closed, ResolvedPublic
Actions

Description

mattflaschen@deployment-bastion:~$ mwscript extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=testwiki --debug --srcpage 'Talk:LQT_2015-07-30' --logfile '/tmp/LQT_2015-07-30.log'
[2015-07-30 21:15:17] Starting LQT conversion of page Talk:LQT_2015-07-30
[2015-07-30 21:15:17] Archiving page from Talk:LQT 2015-07-30 to Talk:LQT 2015-07-30/LQT Archive 1

Warning: Redis::connect(): php_network_getaddresses: getaddrinfo failed: Name or service not known in /mnt/srv/mediawiki-staging/php-master/includes/clientpool/RedisConnectionPool.php on line 235

Warning: Redis::connect(): php_network_getaddresses: getaddrinfo failed: Name or service not known in /mnt/srv/mediawiki-staging/php-master/includes/clientpool/RedisConnectionPool.php on line 235

Warning: Redis::connect(): php_network_getaddresses: getaddrinfo failed: Name or service not known in /mnt/srv/mediawiki-staging/php-master/includes/clientpool/RedisConnectionPool.php on line 235

The warnings repeat (there are already more than that, and it's still running):

[2015-07-30 22:18:34] exception 'MWException' with message 'RedisConnectionPool: invalid configured server ""' in /mnt/srv/mediawiki-staging/php-master/includes/clientpool/RedisConnectionPool.php:222
Stack trace:
#0 /mnt/srv/mediawiki-staging/php-master/includes/objectcache/RedisBagOStuff.php(377): RedisConnectionPool->getConnection(NULL)
#1 /mnt/srv/mediawiki-staging/php-master/includes/objectcache/RedisBagOStuff.php(85): RedisBagOStuff->getConnection('testwiki:abusef...')
#2 /mnt/srv/mediawiki-staging/php-master/extensions/AbuseFilter/AbuseFilter.hooks.php(281): RedisBagOStuff->get('testwiki:abusef...')
#3 [internal function]: AbuseFilterHooks::onGetAutoPromoteGroups(Object(User), Array)
#4 /mnt/srv/mediawiki-staging/php-master/includes/Hooks.php(204): call_user_func_array('AbuseFilterHook...', Array)
#5 /mnt/srv/mediawiki-staging/php-master/includes/Autopromote.php(46): Hooks::run('GetAutoPromoteG...', Array)
#6 /mnt/srv/mediawiki-staging/php-master/includes/User.php(3036): Autopromote::getAutopromoteGroups(Object(User))
#7 /mnt/srv/mediawiki-staging/php-master/includes/User.php(3011): User->getAutomaticGroups(false)
#8 /mnt/srv/mediawiki-staging/php-master/includes/User.php(2981): User->getEffectiveGroups()
#9 /mnt/srv/mediawiki-staging/php-master/includes/User.php(3252): User->getRights()
#10 /mnt/srv/mediawiki-staging/php-master/includes/page/WikiPage.php(2066): User->isAllowed('bot')
#11 /mnt/srv/mediawiki-staging/php-master/includes/page/WikiPage.php(2164): WikiPage->prepareContentForEdit(Object(WikitextContent), Object(Revision), Object(User))
#12 /mnt/srv/mediawiki-staging/php-master/includes/MovePage.php(501): WikiPage->doEditUpdates(Object(Revision), Object(User), Array)
#13 /mnt/srv/mediawiki-staging/php-master/includes/MovePage.php(255): MovePage->moveToInternal(Object(User), Object(Title), 'Conversion of L...', false)
#14 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Import/Converter.php(261): MovePage->move(Object(User), 'Conversion of L...', false)
#15 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Import/Converter.php(183): Flow\Import\Converter->movePage(Object(Title), Object(Title))
#16 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Import/Converter.php(139): Flow\Import\Converter->doConversion(Object(Title), NULL)
#17 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/includes/Import/Converter.php(108): Flow\Import\Converter->convert(Object(Title))
#18 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php(77): Flow\Import\Converter->convertAll(Array)
#19 /mnt/srv/mediawiki-staging/php-master/maintenance/doMaintenance.php(103): ConvertLqtPageOnLocalWiki->execute()
#20 /mnt/srv/mediawiki-staging/php-master/extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php(84): require_once('/mnt/srv/mediaw...')
#21 /mnt/srv/mediawiki-staging/multiversion/MWScript.php(97): require_once('/mnt/srv/mediaw...')
#22 {main}

Details

SubjectRepoBranchLines +/-
Throw exception on falsy server in RedisConnectionPool.mediawiki/coremaster+2 -2
Customize query in gerrit

Related Objects

Event Timeline

• Mattflaschen-WMF created this task.Jul 30 2015, 9:20 PM
• Mattflaschen-WMF raised the priority of this task from to Needs Triage.
• Mattflaschen-WMF updated the task description. (Show Details)
• Mattflaschen-WMF added a project: Beta-Cluster-Infrastructure.
• Mattflaschen-WMF subscribed.
Restricted Application added subscribers: Luke081515, Aklapper. · View Herald TranscriptJul 30 2015, 9:20 PM
• Mattflaschen-WMF claimed this task.Jul 30 2015, 9:39 PM
gerritbot subscribed.Jul 30 2015, 9:40 PM

Change 228154 had a related patch set uploaded (by Mattflaschen):
Throw exception on falsy server in RedisConnectionPool.

https://gerrit.wikimedia.org/r/228154

gerritbot added a project: Patch-For-Review.Jul 30 2015, 9:40 PM
gerritbot added a comment.Jul 30 2015, 9:57 PM

Change 228154 merged by jenkins-bot:
Throw exception on falsy server in RedisConnectionPool.

https://gerrit.wikimedia.org/r/228154

• Forrestbot added projects: MW-1.26-release, WMF-deploy-2015-08-04_(1.26wmf17).Jul 30 2015, 10:00 PM
• Mattflaschen-WMF mentioned this in rMW70e949bdc2bf: Throw exception on falsy server in RedisConnectionPool..Jul 30 2015, 10:15 PM
• Mattflaschen-WMF updated the task description. (Show Details)Jul 30 2015, 10:19 PM
• Mattflaschen-WMF set Security to None.
Restricted Application added a subscriber: Hydriz. · View Herald TranscriptJul 30 2015, 10:19 PM
bd808 subscribed.Jul 30 2015, 11:48 PM
$ sudo -u www-data php /srv/mediawiki/multiversion/MWScript.php extensions/Flow/maintenance/convertLqtPageOnLocalWiki.php --wiki=testwiki --debug --srcpage 'Talk:LQT_2015-07-30' --logfile '/tmp/LQT_2015-07-30.log'
[2015-07-30 23:48:00] Starting LQT conversion of page Talk:LQT_2015-07-30
[2015-07-30 23:48:00] Archiving page from Talk:LQT 2015-07-30 to Talk:LQT 2015-07-30/LQT Archive 1
[2015-07-30 23:48:04] Importing to Talk:LQT 2015-07-30
[2015-07-30 23:48:04] Workflow isNew: true
[2015-07-30 23:48:06] ensureFlowRevision status isOK: true
[2015-07-30 23:48:06] ensureFlowRevision status isGood: true
[2015-07-30 23:48:06] ensureFlowRevision already-existed: false
[2015-07-30 23:48:06] ensureFlowRevision revision ID: 2022, page ID: 973
[2015-07-30 23:48:07] Importing header
[2015-07-30 23:48:10] Imported 2 revisions for header
[2015-07-30 23:48:10] Flow\Import\LiquidThreadsApi\ApiBackend::retrieveThreadData: Did not find thread with conditions: {"thpage":"Talk:LQT 2015-07-30\/LQT Archive 1","thstartid":1}
[2015-07-30 23:48:10] Flow\Import\LiquidThreadsApi\ApiBackend::retrieveThreadData: Did not find thread with conditions: {"thpage":"Talk:LQT 2015-07-30\/LQT Archive 1","thstartid":1}
[2015-07-30 23:48:10] Imported 1 items, failed 0
[2015-07-30 23:48:10] Completed import to Talk:LQT 2015-07-30 from Talk:LQT 2015-07-30/LQT Archive 1
[2015-07-30 23:48:10] Finished LQT conversion of page Talk:LQT_2015-07-30

$ cat /tmp/LQT_2015-07-30.log
{"lqt-api:local:header_for:Talk:LQT 2015-07-30\/LQT Archive 1:rev:2021":"sm360395cvjoydy0","lqt-api:local:header_for:Talk:LQT 2015-07-30\/LQT Archive 1:rev:scripted:a118c6308be838621021b4ca0cf74d13":"sm3620khb8bptwkl","lqt-api:local:header_for:Talk:LQT 2015-07-30\/LQT Archive 1":"sm360395cvjoydy0"}
bd808 added a comment.Jul 30 2015, 11:55 PM

/etc/nutcracker/nutcracker.yml on deployment-bastion is missing config sections. It has

memcached:
   auto_eject_hosts: true
   distribution: ketama
   hash: md5
   listen: 127.0.0.1:11212
   preconnect: true
   redis: false
   server_connections: 2
   server_failure_limit: 3
   server_retry_timeout: 30000
   timeout: 250
   servers:
   - 10.68.16.14:11211:1
   - 10.68.16.15:11211:1

but deployment-mediawiki01 has

mc-unix:
  auto_eject_hosts: true
  distribution: ketama
  hash: md5
  listen: /var/run/nutcracker/nutcracker.sock 0666
  preconnect: true
  server_connections: 1
  server_failure_limit: 3
  server_retry_timeout: 30000
  servers:
    - 10.68.16.14:11211:1
    - 10.68.16.15:11211:1
  timeout: 250
memcached:
  auto_eject_hosts: true
  distribution: ketama
  hash: md5
  listen: 127.0.0.1:11212
  preconnect: true
  server_connections: 1
  server_failure_limit: 3
  server_retry_timeout: 30000
  servers:
    - 10.68.16.14:11211:1
    - 10.68.16.15:11211:1
  timeout: 250
session-redis:
  auto_eject_hosts: true
  distribution: ketama
  hash: md5
  listen: 127.0.0.1:6380
  preconnect: true
  redis: true
  redis_auth: Q6dzak4k9vjYjh341fHS
  server_connections: 1
  server_failure_limit: 3
  server_retry_timeout: 30000
  servers:
    - 10.68.16.177:6379:1
    - 10.68.16.231:6379:1
  timeout: 1000
• Mattflaschen-WMF closed this task as Resolved.Jul 30 2015, 11:57 PM
• Mattflaschen-WMF reassigned this task from • Mattflaschen-WMF to bd808.
ori subscribed.Jul 30 2015, 11:58 PM

This should work now, because I made CommonSettings.php once again consult $sessionRedis as defined in the realm-specific session.php file. But a proper fix would be to get Nutcracker fully working in Beta, which requires fixing the configuration / Puppetization, as @bd808 notes above.

greg mentioned this in T107538: fix nutcracker config in Beta.Jul 31 2015, 12:11 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL