Page MenuHomePhabricator

updateCollation.php on terbium still run code from 1.28.0-wmf.16 against enwiki ( LoadBalancer::reallyOpenConnection: 402+ connections made (master=db1057) LoadBalancer.php line 850 )
Closed, DuplicatePublic

Description

We had all wikis switched to 1.28.0-wmf.17 but logstash shows that terbium is still running code for the previous version: 1.28.0-wmf.16

Seems to solely target enwiki. Might be a cause for:

LoadBalancer::reallyOpenConnection: 402+ connections made (master=db1057)
LoadBalancer.php line 850 calls wfBacktrace()
LoadBalancer.php line 782 calls LoadBalancer->reallyOpenConnection() LoadBalancer.php line 696 calls LoadBalancer->openForeignConnection()
LoadBalancer.php line 487 calls LoadBalancer->openConnection()
LoadBalancer.php line 434 calls LoadBalancer->doWait()
LBFactory.php line 406 calls LoadBalancer->waitForAll()
GlobalFunctions.php line 3323 calls LBFactory->waitForReplication()
Maintenance.php line 1239 calls wfWaitForSlaves()
updateCollation.php line 216 calls Maintenance->commitTransaction()
doMaintenance.php line 106 calls UpdateCollation->execute()
updateCollation.php line 342 calls require_once() MWScript.php line 97 calls require_once()
niharika29 12091  0.0  0.0  12412  1240 pts/24   S+   Aug29   0:00      \_ /bin/bash /usr/local/bin/mwscript maintenance/updateCollation.php --wiki=enwiki --force
root     12096  0.0  0.0  65072  1636 pts/24   S+   Aug29   0:00          \_ sudo -u www-data php5 /srv/mediawiki-staging/multiversion/MWScript.php maintenance/updateCollation.php --wiki=enwiki --force
www-data 12099  7.4  0.1 333612 51224 pts/24   S+   Aug29 384:25              \_ php5 /srv/mediawiki-staging/multiversion/MWScript.php maintenance/updateCollation.php --wiki=enwiki --force

Process started by @Niharika from Community-Tech

Event Timeline

hashar created this task.Sep 2 2016, 8:23 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 2 2016, 8:23 AM
hashar triaged this task as High priority.Sep 2 2016, 8:23 AM
hashar added a comment.Sep 2 2016, 8:34 AM

updateCollation.php is a long living script and its code paths invokes wfWaitForSlaves() which ends up opening a connection. Maybe it keeps disconnecting and on each iteration ends up having to reopen one.

hashar added a comment.Sep 3 2016, 7:15 AM

The too many connections count scared me. It is less of an issue that I originally thought of.

Merging in parent task that has more details.