Page MenuHomePhabricator

Cannot access the database: No working replica DB server: Unknown error
Closed, ResolvedPublic

Description

Rollout of wmf.21 to testwiki caused 503s with the error message, Cannot access the database: No working replica DB server: Unknown error

{
  "args": [],
  "type": "->",
  "class": "LoadBalancer",
  "function": "reportConnectionError",
  "line": 565,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/libs/rdbms/loadbalancer/LoadBalancer.php"
},
{
  "args": [
    "boolean",
    "array",
    "boolean"
  ],
  "type": "->",
  "class": "LoadBalancer",
  "function": "getConnection",
  "line": 3073,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/GlobalFunctions.php"
},
{
  "args": [
    "integer"
  ],
  "function": "wfGetDB",
  "line": 739,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/user/User.php"
},
{
  "args": [
    "string"
  ],
  "type": "::",
  "class": "User",
  "function": "idFromName",
  "line": 1839,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/extensions/CentralAuth/includes/CentralAuthUser.php"
},
{
  "args": [],
  "type": "->",
  "class": "CentralAuthUser",
  "function": "canAuthenticate",
  "line": 1888,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/extensions/CentralAuth/includes/CentralAuthUser.php"
},
{
  "args": [
    "string"
  ],
  "type": "->",
  "class": "CentralAuthUser",
  "function": "authenticateWithToken",
  "line": 179,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/extensions/CentralAuth/includes/session/CentralAuthSessionProvider.php"
},
{
  "args": [
    "WebRequest"
  ],
  "type": "->",
  "class": "CentralAuthSessionProvider",
  "function": "provideSessionInfo",
  "line": 486,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/session/SessionManager.php"
},
{
  "args": [
    "WebRequest"
  ],
  "type": "->",
  "class": "MediaWiki\\Session\\SessionManager",
  "function": "getSessionInfoForRequest",
  "line": 189,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/session/SessionManager.php"
},
{
  "args": [
    "WebRequest"
  ],
  "type": "->",
  "class": "MediaWiki\\Session\\SessionManager",
  "function": "getSessionForRequest",
  "line": 735,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/WebRequest.php"
},
{
  "args": [],
  "type": "->",
  "class": "WebRequest",
  "function": "getSession",
  "line": 128,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/session/SessionManager.php"
},
{
  "args": [],
  "type": "::",
  "class": "MediaWiki\\Session\\SessionManager",
  "function": "getGlobalSession",
  "line": 759,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/Setup.php"
},
{
  "args": [
    "string"
  ],
  "function": "include",
  "line": 137,
  "file": "/srv/mediawiki/php-1.28.0-wmf.21/includes/WebStart.php"
},
{
  "args": [
    "string"
  ],
  "function": "include",
  "line": 3,
  "file": "/srv/mediawiki/w/favicon.php"
}

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 4 2016, 8:49 PM
greg triaged this task as Unbreak Now! priority.Oct 4 2016, 8:51 PM
Restricted Application added subscribers: Jay8g, Luke081515, TerraCodes. · View Herald TranscriptOct 4 2016, 8:51 PM

Change 314173 had a related patch set uploaded (by Aaron Schulz):
Add version to LoadMonitor::getCacheKey()

https://gerrit.wikimedia.org/r/314173

Change 314176 had a related patch set uploaded (by Thcipriani):
Add version to LoadMonitor::getCacheKey()

https://gerrit.wikimedia.org/r/314176

Change 314173 merged by jenkins-bot:
Add version to LoadMonitor::getCacheKey()

https://gerrit.wikimedia.org/r/314173

Change 314176 merged by jenkins-bot:
Add version to LoadMonitor::getCacheKey()

https://gerrit.wikimedia.org/r/314176

Mentioned in SAL (#wikimedia-operations) [2016-10-04T21:34:46Z] <thcipriani@tin> Synchronized php-1.28.0-wmf.21/includes/libs/rdbms/loadmonitor/LoadMonitor.php: [[gerrit:314176|Add version to LoadMonitor::getCacheKey() (T147359)]] (duration: 00m 53s)

thcipriani closed this task as Resolved.Oct 4 2016, 9:57 PM
thcipriani assigned this task to aaron.

1.28.0-wmf.21 is now on group0 wikis and I'm no longer seeing this error. Marking as resolved.

thcipriani reopened this task as Open.Oct 5 2016, 5:40 PM

Spoke too soon. Although the rate at which this error is occurring has fallen considerably, this error is still in the logs.

Change 314340 had a related patch set uploaded (by Aaron Schulz):
Make LoadMonitor use $serverIndexes in the cache key

https://gerrit.wikimedia.org/r/314340

Change 314414 had a related patch set uploaded (by Thcipriani):
Make LoadMonitor use $serverIndexes in the cache key

https://gerrit.wikimedia.org/r/314414

Change 314340 merged by jenkins-bot:
Make LoadMonitor use $serverIndexes in the cache key

https://gerrit.wikimedia.org/r/314340

Change 314414 merged by jenkins-bot:
Make LoadMonitor use $serverIndexes in the cache key

https://gerrit.wikimedia.org/r/314414

Mentioned in SAL (#wikimedia-operations) [2016-10-05T21:08:15Z] <thcipriani@tin> Synchronized php-1.28.0-wmf.21/maintenance/lag.php: [[gerrit:314414|Make LoadMonitor use $serverIndexes in the cache key (T147359)]] PART I (duration: 00m 50s)

Mentioned in SAL (#wikimedia-operations) [2016-10-05T21:09:20Z] <thcipriani@tin> Synchronized php-1.28.0-wmf.21/includes/libs/rdbms: [[gerrit:314414|Make LoadMonitor use $serverIndexes in the cache key (T147359)]] PART II (duration: 00m 55s)

thcipriani closed this task as Resolved.Oct 6 2016, 10:49 PM

Seems to be fully gone from the logs afaict

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:10 PM