Page MenuHomePhabricator

Database error: "SqlBagOStuff::set". Database returned error "1114: The table 'pc233' is full (10.0.6.50)".
Closed, ResolvedPublic

Description

Just saving an edit:
http://meta.wikimedia.org/w/index.php?title=Talk:Wikidata&action=submit

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:
(SQL query hidden)
from within function "SqlBagOStuff::set". Database returned error "1114: The table 'pc233' is full (10.0.6.50)".


Version: unspecified
Severity: critical

Details

Reference
bz38202

Event Timeline

bzimport raised the priority of this task from to Unbreak Now!.Nov 22 2014, 12:57 AM
bzimport set Reference to bz38202.
bzimport added a subscriber: Unknown Object (MLST).

Not just when saving, here, reading a revision:

http://meta.wikimedia.org/w/index.php?title=Talk%3AWikidata&diff=3879256&oldid=3879182

Database error
A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:
(SQL query hidden)
from within function "SqlBagOStuff::set". Database returned error "1114: The table 'pc233' is full (10.0.6.50)".

Not just pc233:

The table 'pc193' is full (10.0.6.50)".

I got it on other table :
from within function "SqlBagOStuff::set". Database returned error "1114: The table 'pc193' is full (10.0.6.50)".

https://he.wikipedia.org/w/index.php?title=%D7%A9%D7%99%D7%97%D7%AA_%D7%9E%D7%A9%D7%AA%D7%9E%D7%A9:%D7%A2%D7%A8%D7%9F&action=submit

  • Bug 38203 has been marked as a duplicate of this bug. ***

upload as well, table pc006
upload failed from within function "SqlBagOStuff::set". Database returned error "1114: The table 'pc006' is full (10.0.6.50)".

Seen on at least:

  • metawiki
  • commonswiki
  • hewiki
  • ptwiki
  • itwiki

(all s7 cluster wikis)

So far all mention only 10.0.6.50 (db40, s7 cluster), though the pc### table varies.

It's affecting all wikis because db40 runs the parser cache. Please do not pile further "this wiki too" comments.

  • Bug 38204 has been marked as a duplicate of this bug. ***

content hidden as private in Bugzilla

db40 just went down entirely. Now all reading is off for most/all wikis as well (including en.wikipedia.org).

Skin-less blank db error etc.:

Sorry! This site is experiencing technical difficulties.
Try waiting a few minutes and reloading.
(Cannot contact the database server: Unknown error (10.0.6.50))

vigorous_action wrote:

also jawiki, jaws,jawb.

(In reply to comment #15)

also jawiki, jaws,jawb.

Also ~800 other wikis. Please see comment 10.

Christa.Mu3ller wrote:

funny error message in dewiki on read access:

"Error 500 Internal Server Error Internal Server Error
Guru Meditation: XID: 3696570436 Varnish cache server"

(In reply to comment #18)

Christa, unrelated issue.

Probably not unrelated, actually. db40 went down since ~ comment 14. And all requests that don't hit squid cache server skin-less db error pages (which head with HTTP 500 Internal Server Error).

bits varnish hitting the backend might get that too from the mediawiki/apaches.

(Cite wikisal July 5)

14:07 logmsgbot: midom synchronized wmf-config/InitialiseSettings.php 'disabling parser cache for now'

Browsing is back up for everybody. Logged-in users bypassing parser cache. Logged-out users just hitting squids as usual.

saibotrash wrote:

It seems some links of files to categories (files are not shown on category pages even while being logged in) are broken: https://commons.wikimedia.org/wiki/Commons:Village_pump#Database_error ([https://commons.wikimedia.org/w/index.php?title=Commons:Village_pump&oldid=73829498#Database_error perm.]) A null edit to the file pages helps. Maybe that (or a subset of the triggered actions) can be done to all affected files automatically.

Hi everyone, Domas got things back up and running earlier today. We're still have some work to do to make sure we don't have a similar outage again which we're sorting out now, but the bug as reported was fixed this morning.