At first I thought a news letter was wrongly posted at my own Flow board. Deleted it, and then noticed it was not on my Flow board but at H@r@lds Flow board, and reopened the topic. Then I tried to make a new notice to H@r@ld and got an error.
So, whats goes on? Is this a bug or is it a weird corner case that just happen once?
Full error message:
2017-05-03 16:56:12 [WQoLqwpAAE0AAHp5nt0AAAAH] mw1282 nowiki 1.29.0-wmf.21 exception ERROR: [WQoLqwpAAE0AAHp5nt0AAAAH] /w/api.php Wikimedia\Rdbms\DBConnectionError from line 769 of /srv/mediawiki/php-1.29.0-wmf.21/includes/libs/rdbms/database/Database.php: Cannot access the database: Too many connections (10.64.16.20) {"exception_id":"WQoLqwpAAE0AAHp5nt0AAAAH","caught_by":"other"}
[Exception Wikimedia\Rdbms\DBConnectionError] (/srv/mediawiki/php-1.29.0-wmf.21/includes/libs/rdbms/database/Database.php:769) Cannot access the database: Too many connections (10.64.16.20)
#0 /srv/mediawiki/php-1.29.0-wmf.21/includes/libs/rdbms/loadbalancer/LoadBalancer.php(995): Wikimedia\Rdbms\Database->reportConnectionError(string)
#1 /srv/mediawiki/php-1.29.0-wmf.21/includes/libs/rdbms/loadbalancer/LoadBalancer.php(666): Wikimedia\Rdbms\LoadBalancer->reportConnectionError()
#2 /srv/mediawiki/php-1.29.0-wmf.21/extensions/Flow/includes/DbFactory.php(53): Wikimedia\Rdbms\LoadBalancer->getConnection(integer, array, string)
#3 /srv/mediawiki/php-1.29.0-wmf.21/extensions/Flow/includes/SubmissionHandler.php(126): Flow\DbFactory->getDB(integer)
#4 /srv/mediawiki/php-1.29.0-wmf.21/extensions/Flow/includes/WorkflowLoader.php(66): Flow\SubmissionHandler->commit(Flow\Model\Workflow, array)
#5 /srv/mediawiki/php-1.29.0-wmf.21/extensions/Flow/includes/Api/ApiFlowBasePost.php(35): Flow\WorkflowLoader->commit(array)
#6 /srv/mediawiki/php-1.29.0-wmf.21/extensions/Flow/includes/Api/ApiFlow.php(98): Flow\Api\ApiFlowBasePost->execute()
#7 /srv/mediawiki/php-1.29.0-wmf.21/includes/api/ApiMain.php(1578): Flow\Api\ApiFlow->execute()
#8 /srv/mediawiki/php-1.29.0-wmf.21/includes/api/ApiMain.php(545): ApiMain->executeAction()
#9 /srv/mediawiki/php-1.29.0-wmf.21/includes/api/ApiMain.php(516): ApiMain->executeActionWithErrorHandling()
#10 /srv/mediawiki/php-1.29.0-wmf.21/api.php(83): ApiMain->execute()
#11 /srv/mediawiki/w/api.php(3): include(string)
#12 {main}It appears this is caused by an x1 outage, which ops is already addressing: https://gerrit.wikimedia.org/r/#/c/351679/
I would say the chance is pretty high for an user error on the initial deletion, probably the user (me) was tired, but the second error was real enough. I have although successfully posted a message on the users Flow board. That message is w:no:Sak:Tpuxh1fq9e6u0gww.
jeblad your error was real, and it was caused by the incident at T164407, which has been since mitigated. You should not have any further issue from now on, but please tell us if otherwise.
It wasn't you, it was us :) the database servers that Flow uses were overloaded because Cognate was misbehaving, see T164407: Cognate has been disabled from WMF because it caused an outage on x1 by overtaking 10000 concurrent connections. @jcrespo disabled Cognate at 17:14 UTC (shortly after my previous comment), and Flow should be working again now.