Page MenuHomePhabricator

labswiki cannot connect to x1-slave (db1031), and soon, x1-master, either [Error connecting to 10.64.16.20: :real_connect(): (HY000/2003): Can't connect to MySQL server on '10.64.16.20' (4)]
Closed, ResolvedPublicPRODUCTION ERROR

Description

Should labswiki connect to x1 shard?

If not, reconfigure it so that it queries local tables.
If yes, puppetize an appropiate hole on the mysql firewalls.

Details

Event Timeline

• jcrespo raised the priority of this task from to Needs Triage.
• jcrespo updated the task description. (Show Details)
• jcrespo added projects: SRE, Cloud-Services.
• jcrespo subscribed.
{
  "_index": "logstash-2015.12.18",
  "_type": "mediawiki",
  "_id": "AVG12myMMRv_gmyx2RWa",
  "_score": null,
  "_source": {
    "message": "Error connecting to 10.64.16.20: :real_connect(): (HY000/2003): Can't connect to MySQL server on '10.64.16.20' (4)",
    "@version": 1,
    "@timestamp": "2015-12-18T16:09:28.000Z",
    "type": "mediawiki",
    "host": "silver",
    "level": "ERROR",
    "tags": [
      "syslog",
      "es",
      "es",
      "normalized_message_untrimmed"
    ],
    "channel": "wfLogDBError",
    "url": "/wiki/Special:WhatLinksHere/OpenStack",
    "ip": "<my ip>",
    "http_method": "GET",
    "server": "wikitech.wikimedia.org",
    "referrer": null,
    "uid": "0cfa291",
    "process_id": 27346,
    "wiki": "labswiki",
    "db_server": "10.64.16.20",
    "db_name": "flowdb",
    "db_user": "wikiuser",
    "method": "DatabaseMysqlBase::open",
    "error": ":real_connect(): (HY000/2003): Can't connect to MySQL server on '10.64.16.20' (4)",
    "normalized_message": "Error connecting to 10.64.16.20: :real_connect(): (HY000/2003): Can't connect to MySQL server on '10.64.16.20' (4)"
  },
  "sort": [
    1450454968000
  ]
}

Probably the answer to this is to just disable flow. Alternatively we could host flow with the local wikitech db, but it doesn't sound like anyone has been missing it so far.

Change 264988 had a related patch set uploaded (by Andrew Bogott):
Disable flow on wikitech

https://gerrit.wikimedia.org/r/264988

RobH mentioned this in Unknown Object (Task).Apr 19 2016, 5:21 PM
RobH mentioned this in Unknown Object (Task).
• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:11 PM