Maniphest T126942

labs -Database is offline
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Boshomi
	Feb 14 2016, 10:54 PM

Tags

Referenced Files

None

Subscribers

View All 9 Subscribers

Description

look at https://tools.wmflabs.org/replag/ => 500 Server Internal Error

or results like this

https://tools.wmflabs.org/templatecount/index.php?lang=de&namespace=10&name=Cite+web

Warning: mysqli::mysqli(): (HY000/2003): Can't connect to MySQL server on 'dewiki.labsdb' (111) in /data/project/jarry-common/public_html/libs/database.php on line 27 Call Stack: 0.0130 710128 1. {main}() /data/project/templatecount/public_html/index.php:0 0.7229 3510720 2. dbconnect() /data/project/templatecount/public_html/index.php:52 0.7320 3515320 3. mysqli->mysqli() /data/project/jarry-common/public_html/libs/database.php:27

Details

	Subject	Repo	Branch	Lines +/-
	Depool labsdb1002	operations/puppet	production	+6 -5

Customize query in gerrit

Related Objects

Mentioned In: Blog Post: New Wiki Replica servers ready for use
T126946: disk failure on labsdb1002
T126943: ERROR 2003 (HY000): Can't connect to MySQL server

Event Timeline

Boshomi created this task.Feb 14 2016, 10:54 PM

Boshomi raised the priority of this task from to Needs Triage.

Boshomi updated the task description. (Show Details)

Boshomi subscribed.

Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptFeb 14 2016, 10:54 PM

known, being worked on. one of the DB server's drives has failed

labsdb1002 has a broken disk which resulted in the partition with the db being off line. Folks will be redirecting queries to other servers shortly.

Change 270650 had a related patch set uploaded (by Jcrespo):
Depool labsdb1002

https://gerrit.wikimedia.org/r/270650

gerritbot added a project: Patch-For-Review.Feb 14 2016, 10:59 PM

Change 270650 merged by Andrew Bogott:
Depool labsdb1002

https://gerrit.wikimedia.org/r/270650

Josve05a mentioned this in T126943: ERROR 2003 (HY000): Can't connect to MySQL server.Feb 14 2016, 11:07 PM

Krenair merged a task: T126943: ERROR 2003 (HY000): Can't connect to MySQL server.Feb 14 2016, 11:07 PM

Krenair added subscribers: Josve05a, Fae.

• chasemp mentioned this in T126946: disk failure on labsdb1002.Feb 14 2016, 11:15 PM

this should be worked around now with https://gerrit.wikimedia.org/r/#/c/270650/

replag and templatecount have been restarted.

bd808 mentioned this in Blog Post: New Wiki Replica servers ready for use.Sep 22 2017, 6:52 PM