Receiving "Failed to create a user log table to use. This table is vital for the operation of this interface. Exiting..." on single page run
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
This appears to be a DB problem. This is making the tool and the bot notoriously unreliable. IABot workers are constantly crashing on different wikis due to DB failures.
Comment Actions
Can you give some more details about this?
Where is this happening?
Where are you trying to connect to?
Which user are you using to connect to the DB?
Comment Actions
This is happening with InternetArchiveBot and https://tools.wmflabs.org/iabot
Because the DB is vital to IABot and it's interface, and a failure can result to radically unpredictable onwiki behavior, a safety I built in terminates execution with error messages.
The DB in question is s51059__cyberbot connecting with user s51059.
Comment Actions
Indeed - we saw that user on labsdb1005 while debugging T178272.
Is this still happening?
Comment Actions
We had to limit that user to 10 connections as we were not sure it was causing the issues with the server.
Right now those 10 connections are open and thus it is probably getting errors because of that.
Those connections are not doing anything, just sleeping.
Ideally it should not leave lots of connections opens, it should just open a connection, use it, and close it. This server is shared, so we need to limit resources,.
I have removed the limit and I can see that: https://tools.wmflabs.org/iabot/ works again.
As I said, can you please try to change it in order to open-close connections, otherwise, whenever we have to limit it (maybe not to 10, but to some other value) it will fail.
Comment Actions
Could it mean someone is abusing the bot usage? Does it depend on recentchanges or user requests? Is there a limit of "usages" than anyone can do?
Comment Actions
You'll find those connections aren't constantly sleeping. It makes high speed read and write queries every second.
Comment Actions
Some of them were sleeping for a long time:
| 3792155 | root | localhost | s51059__cyberbot | Sleep | 22454 | | NULL | 0.000 | | 4700688 | s51059 | 10.68.23.85:59967 | s51059__cyberbot | Sleep | 317 | | NULL | 0.000 | | 4700707 | s51059 | 10.68.23.85:59983 | s51059__cyberbot | Sleep | 318 |
Comment Actions
Hmm... IABot doesn't leave connections to idle. Where are they coming from. I do have Workbench open on another machine.
Comment Actions
"Usage of connection pools (maintaining open connections without them being in use), persistent connections, or any kind of connection pattern that maintains several connections open even if they are unused is not permitted on shared MySQL instances (replicas and toolsdb)."
This is why I asked that, if those are not intended, maybe someone else (not the owner) is "abusing" the tools usage. It happens sometimes, 3rd party/indirect users misuse the original code.
Comment Actions
It started to work after I removed the limit of 10 connections, so it was clearly trying to create more than 10 at the time. So far it is fine, but if we see that the host load starts to bump again, we might need to limit it.
This might not even be the cause of the server issue anyways and it happened to be triaged while we were trying to get the server back stable :-)
Comment Actions
And I thought it was harder on the server to constantly reconnect when making a query. Where are those two connections coming from?
Comment Actions
And I thought it was harder on the server to constantly reconnect
If this was a dedicated server, pooling would be preferred. Being a highly-shared database, it is better to disconnect ASAP. The "rule" is to enforce that. Not doing that can make the server incompatible in the future- so better adapt now.
Alternative, you can consider asking for dedicated resources- it may be possible with soon to be installed new database hardware (but obviously, we cannot guarantee to be granted; but I would personally would back that request).
Comment Actions
Well the DB is designed very efficiently, thanks to Chase. He helped me to setup very well indexed tables so read and write queries make no noticeable impact on load.
Comment Actions
I think a dedicated DB would be ideal. As IABot expands to more wikis, more concurrent connections will be opened. Even if they are immediately opened and closed, it will still exceed 10 connections at a time. So I think a dedicated instance would be best.
Comment Actions
I'm rapidly hitting refresh and getting "Failed to create a user log table to use. This table is vital for the operation of this interface. Exiting..." every time.
Comment Actions
2017-10-16 17:01:25: (mod_fastcgi.c.2673) FastCGI-stderr: PHP Warning: mysqli_connect(): (HY000/1226): User 's51059' has exceeded the 'max_user_connections' resource (current value: 10) in /mnt/nfs/labstore-secondary-tools-project/iabot/public_html/Includes/DB2.php on line 27
Comment Actions
Thank you. What do I do to request a dedicated DB instance for Cyberbot/IABot? While even 30 simultaneous queries shouldn't spike the load on the DB significantly, it would probably be preferable to move the DB to a dedicated VM as IABot expands to more wikis.
Comment Actions
Cyberpower678 file a ticket to wmcs Data services, and CC me. Do not expect it to be quick. For the time being, you should try to stick to 10 threads- if the server crashes and stops giving service, that will be much worse than having less available connections. If the server crashes again, the limit will be reinstated indefinitely- not only to you, to all users.
Comment Actions
The process for that would be to request a Cloud VPS project and take ownership of your own infrastructure. We do not have the resources in the Cloud Services team nor the DBA team to manage one-off data storage systems for projects that outgrow our shared infrastructure. There will be work starting in January or so to move the existing tools.labsdb services to new hardware, but I don't think we have any forecasts at this time of what increase in performance or capability that may provide.
Comment Actions
I can try, but I can't guarantee it. Like I said, I was graciously assisted when setting up the DB as to make almost no impact on DB load. @Volans was the one who assisted me with that and confirmed himself that the load was negligible.
I would be glad to move it to my own VPS, but I have no experience maintaining a DB, or even setting it up. I would need some assistance with that.
Comment Actions
with that and confirmed himself that the load was negligible
https://grafana.wikimedia.org/dashboard/file/server-board.json?refresh=1m&panelId=19&fullscreen&orgId=1&var-server=labsdb1005&var-network=bond0&from=1507570719541&to=1508175519541 11:00am- 10 connection limit is set (which rules say you shouldn't even use 10 connections all the time).
@Volans can be very helpful, but the problem is with us DBAs and cloud team. Your service is taking more load than all other thousand users together. We are not happy about that.
Comment Actions
I do find that somewhat difficult to believe. When the new DB was tested, that was with 28 concurrent connections and the load didn’t go up. Looking at the past 30 days, where the bot has consistently been running the same queries throughout I see the load going up and down. Something here just doesn’t line up with my line of logic. Don’t misunderstand. I’m not being confrontational, just trying to understand. IABot, even with the number of connections, should not be spiking the load that significantly. Are there any queries you can point me to that maybe I can optimize?
Comment Actions
tools.cyberbot (s51059) is currently the number 5 user of disk space on tools.labsdb (49.6 GB). The externallinks_global table has grown to 36G which may have changed the performance footprint from the last time that we dug into this tool's database usage. More tools are migrating their data to tools.labsdb as well due to the impending shutdown of c1.labsdb and c3.labsdb which have historically been used by some tools for persistent data storage. This is a round about way of saying that these are dynamic systems and what worked yesterday may or may not work today or tomorrow.
Here's an example of a slow query as reported by tendril:
UPDATE `externallinks_global` SET `archive_url` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN 'https://web.archive.org/web/20121004185041/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2001.pdf' WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN 'https://web.archive.org/web/20160106225302/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2005/405.pdf' WHEN '24763990' THEN NULL WHEN '17860024' THEN 'https://web.archive.org/web/20120418033136/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2005.pdf' WHEN '17858907' THEN NULL WHEN '27383037' THEN 'https://web.archive.org/web/20160530000407/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2005/1029.pdf' WHEN '34672488' THEN NULL WHEN '17858939' THEN 'https://web.archive.org/web/20121005193544/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2006/904.pdf' WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN 'https://web.archive.org/web/20120417030418/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2006.pdf' WHEN '17858936' THEN 'https://web.archive.org/web/20121005192928/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2006/901.pdf' WHEN '24766541' THEN NULL WHEN '17859950' THEN 'https://web.archive.org/web/20120417223012/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Doubles_Numeric_2006.pdf' WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN 'https://web.archive.org/web/20160426102220/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2007/408.pdf' WHEN '30732610' THEN 'https://web.archive.org/web/20120417095810/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2007.pdf' WHEN '17859014' THEN 'https://web.archive.org/web/20110811090700/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2008/904.pdf' WHEN '17859015' THEN NULL WHEN '17860026' THEN 'https://web.archive.org/web/20120417114832/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2008.pdf' WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN 'https://web.archive.org/web/20121005194955/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/901.pdf' WHEN '17859022' THEN 'https://web.archive.org/web/20121005195304/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/709.pdf' WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN 'https://web.archive.org/web/20121003231508/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/905.pdf' WHEN '23258189' THEN NULL WHEN '28558711' THEN 'https://web.archive.org/web/20120417065116/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2009.pdf' WHEN '30427325' THEN NULL WHEN '27419864' THEN 'https://web.archive.org/web/20130627012056/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2010/904.pdf' WHEN '19087552' THEN NULL WHEN '17859036' THEN 'https://web.archive.org/web/20110626215531/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2010/905.pdf' WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN 'https://web.archive.org/web/20120905140808/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2010.pdf' WHEN '17859043' THEN NULL WHEN '27419867' THEN 'https://web.archive.org/web/20130914210143/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2011/904.pdf' WHEN '17859042' THEN 'https://web.archive.org/web/20130912130210/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2011/901.pdf' WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN 'https://web.archive.org/web/20150318062445/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2011.pdf' WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN 'https://web.archive.org/web/20130626162337/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2012/904.pdf' WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN 'https://web.archive.org/web/20130423043447/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2012.pdf' WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN 'https://web.archive.org/web/20130425220412/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Doubles_Numeric_2012.pdf' WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `has_archive` = CASE `url_id` WHEN '31567734' THEN '0' WHEN '29358967' THEN '0' WHEN '17860022' THEN '1' WHEN '17860023' THEN '0' WHEN '34672487' THEN '0' WHEN '24762045' THEN '0' WHEN '19312685' THEN '0' WHEN '17858911' THEN '0' WHEN '27383264' THEN '1' WHEN '24763990' THEN '0' WHEN '17860024' THEN '1' WHEN '17858907' THEN '0' WHEN '27383037' THEN '1' WHEN '34672488' THEN '0' WHEN '17858939' THEN '1' WHEN '27382945' THEN '0' WHEN '27382948' THEN '0' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '0' WHEN '17859950' THEN '1' WHEN '27382914' THEN '0' WHEN '27382946' THEN '0' WHEN '17858946' THEN '0' WHEN '17858948' THEN '0' WHEN '17858948' THEN '0' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '0' WHEN '17860026' THEN '1' WHEN '17858989' THEN '0' WHEN '17858994' THEN '0' WHEN '27383500' THEN '0' WHEN '27383085' THEN '0' WHEN '27383738' THEN '0' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '0' WHEN '17859017' THEN '0' WHEN '28558712' THEN '0' WHEN '17859029' THEN '1' WHEN '23258189' THEN '0' WHEN '28558711' THEN '1' WHEN '30427325' THEN '0' WHEN '27419864' THEN '1' WHEN '19087552' THEN '0' WHEN '17859036' THEN '1' WHEN '33750214' THEN '0' WHEN '27383447' THEN '0' WHEN '27383830' THEN '0' WHEN '28574522' THEN '0' WHEN '17860027' THEN '1' WHEN '17859043' THEN '0' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '0' WHEN '17859951' THEN '0' WHEN '17860028' THEN '1' WHEN '25866129' THEN '0' WHEN '27383509' THEN '0' WHEN '27419869' THEN '1' WHEN '27276338' THEN '0' WHEN '6307332' THEN '0' WHEN '38883405' THEN '0' WHEN '36243487' THEN '0' WHEN '27315872' THEN '0' WHEN '17860029' THEN '1' WHEN '17859047' THEN '0' WHEN '27383131' THEN '0' WHEN '27383696' THEN '0' WHEN '17859952' THEN '1' WHEN '27383293' THEN '0' WHEN '27383200' THEN '0' WHEN '27383199' THEN '0' WHEN '27382961' THEN '0' END, `live_state` = CASE `url_id` WHEN '31567734' THEN '1' WHEN '29358967' THEN '1' WHEN '17860022' THEN '0' WHEN '17860023' THEN '1' WHEN '34672487' THEN '1' WHEN '24762045' THEN '1' WHEN '19312685' THEN '1' WHEN '17858911' THEN '5' WHEN '27383264' THEN '5' WHEN '24763990' THEN '5' WHEN '17860024' THEN '0' WHEN '17858907' THEN '5' WHEN '27383037' THEN '5' WHEN '34672488' THEN '1' WHEN '17858939' THEN '0' WHEN '27382945' THEN '5' WHEN '27382948' THEN '5' WHEN '17860025' THEN '0' WHEN '17858936' THEN '0' WHEN '24766541' THEN '1' WHEN '17859950' THEN '0' WHEN '27382914' THEN '5' WHEN '27382946' THEN '5' WHEN '17858946' THEN '5' WHEN '17858948' THEN '5' WHEN '17858948' THEN '5' WHEN '17858963' THEN '5' WHEN '30732610' THEN '0' WHEN '17859014' THEN '0' WHEN '17859015' THEN '5' WHEN '17860026' THEN '0' WHEN '17858989' THEN '5' WHEN '17858994' THEN '5' WHEN '27383500' THEN '5' WHEN '27383085' THEN '5' WHEN '27383738' THEN '5' WHEN '17859025' THEN '5' WHEN '17859022' THEN '5' WHEN '17859027' THEN '5' WHEN '17859017' THEN '5' WHEN '28558712' THEN '1' WHEN '17859029' THEN '5' WHEN '23258189' THEN '5' WHEN '28558711' THEN '0' WHEN '30427325' THEN '1' WHEN '27419864' THEN '0' WHEN '19087552' THEN '5' WHEN '17859036' THEN '5' WHEN '33750214' THEN '1' WHEN '27383447' THEN '5' WHEN '27383830' THEN '5' WHEN '28574522' THEN '1' WHEN '17860027' THEN '0' WHEN '17859043' THEN '5' WHEN '27419867' THEN '0' WHEN '17859042' THEN '0' WHEN '19087566' THEN '5' WHEN '17859951' THEN '1' WHEN '17860028' THEN '0' WHEN '25866129' THEN '5' WHEN '27383509' THEN '5' WHEN '27419869' THEN '0' WHEN '27276338' THEN '5' WHEN '6307332' THEN '5' WHEN '38883405' THEN '3' WHEN '36243487' THEN '3' WHEN '27315872' THEN '5' WHEN '17860029' THEN '0' WHEN '17859047' THEN '5' WHEN '27383131' THEN '5' WHEN '27383696' THEN '5' WHEN '17859952' THEN '0' WHEN '27383293' THEN '5' WHEN '27383200' THEN '5' WHEN '27383199' THEN '5' WHEN '27382961' THEN '5' END, `last_deadCheck` = CASE `url_id` WHEN '31567734' THEN '2017-10-16 17:37:00' WHEN '29358967' THEN '2017-10-16 17:37:00' WHEN '17860022' THEN '2017-10-16 17:37:00' WHEN '17860023' THEN '2017-10-16 17:37:00' WHEN '34672487' THEN '2017-10-16 17:37:00' WHEN '24762045' THEN '2017-10-16 17:37:00' WHEN '19312685' THEN '2017-10-16 17:37:00' WHEN '17858911' THEN '2017-06-17 14:09:06' WHEN '27383264' THEN '0000-00-00 00:00:00' WHEN '24763990' THEN '2017-06-17 13:43:19' WHEN '17860024' THEN '2017-10-16 17:37:00' WHEN '17858907' THEN '2017-06-17 12:43:38' WHEN '27383037' THEN '0000-00-00 00:00:00' WHEN '34672488' THEN '2017-10-16 17:37:00' WHEN '17858939' THEN '2017-06-17 18:27:09' WHEN '27382945' THEN '0000-00-00 00:00:00' WHEN '27382948' THEN '0000-00-00 00:00:00' WHEN '17860025' THEN '2017-09-23 03:05:09' WHEN '17858936' THEN '2017-06-08 09:20:00' WHEN '24766541' THEN '2017-10-16 17:37:00' WHEN '17859950' THEN '2017-10-16 17:37:00' WHEN '27382914' THEN '0000-00-00 00:00:00' WHEN '27382946' THEN '0000-00-00 00:00:00' WHEN '17858946' THEN '2017-06-17 21:00:24' WHEN '17858948' THEN '2017-06-17 21:39:43' WHEN '17858948' THEN '2017-06-17 21:39:43' WHEN '17858963' THEN '2017-06-17 20:24:14' WHEN '30732610' THEN '2017-10-16 17:37:00' WHEN '17859014' THEN '2017-06-18 06:20:48' WHEN '17859015' THEN '2017-06-18 06:08:16' WHEN '17860026' THEN '2017-10-16 17:37:00' WHEN '17858989' THEN '2017-06-18 02:52:40' WHEN '17858994' THEN '2017-06-18 03:35:39' WHEN '27383500' THEN '0000-00-00 00:00:00' WHEN '27383085' THEN '0000-00-00 00:00:00' WHEN '27383738' THEN '0000-00-00 00:00:00' WHEN '17859025' THEN '2017-06-18 10:06:01' WHEN '17859022' THEN '2017-06-18 12:00:06' WHEN '17859027' THEN '2017-06-08 09:20:00' WHEN '17859017' THEN '2017-06-08 09:20:00' WHEN '28558712' THEN '2017-10-16 17:37:00' WHEN '17859029' THEN '2017-06-18 14:39:25' WHEN '23258189' THEN '2017-05-15 02:31:30' WHEN '28558711' THEN '2017-09-23 03:05:09' WHEN '30427325' THEN '2017-10-16 17:37:00' WHEN '27419864' THEN '2017-10-05 16:26:26' WHEN '19087552' THEN '2017-06-08 09:20:00' WHEN '17859036' THEN '2017-06-19 00:28:15' WHEN '33750214' THEN '2017-10-16 17:37:00' WHEN '27383447' THEN '0000-00-00 00:00:00' WHEN '27383830' THEN '0000-00-00 00:00:00' WHEN '28574522' THEN '2017-10-16 17:37:00' WHEN '17860027' THEN '2017-09-22 21:26:57' WHEN '17859043' THEN '2017-06-19 07:31:25' WHEN '27419867' THEN '2017-10-05 16:26:31' WHEN '17859042' THEN '2017-10-16 17:37:00' WHEN '19087566' THEN '2017-06-08 09:20:00' WHEN '17859951' THEN '2017-10-16 17:37:00' WHEN '17860028' THEN '2017-10-11 04:21:25' WHEN '25866129' THEN '0000-00-00 00:00:00' WHEN '27383509' THEN '0000-00-00 00:00:00' WHEN '27419869' THEN '2017-10-05 16:26:36' WHEN '27276338' THEN '0000-00-00 00:00:00' WHEN '6307332' THEN '0000-00-00 00:00:00' WHEN '38883405' THEN '2017-10-15 02:47:31' WHEN '36243487' THEN '2017-10-16 17:37:00' WHEN '27315872' THEN '0000-00-00 00:00:00' WHEN '17860029' THEN '2017-10-16 17:37:00' WHEN '17859047' THEN '2016-11-05 12:25:00' WHEN '27383131' THEN '0000-00-00 00:00:00' WHEN '27383696' THEN '0000-00-00 00:00:00' WHEN '17859952' THEN '2017-10-16 17:37:00' WHEN '27383293' THEN '0000-00-00 00:00:00' WHEN '27383200' THEN '0000-00-00 00:00:00' WHEN '27383199' THEN '0000-00-00 00:00:00' WHEN '27382961' THEN '0000-00-00 00:00:00' END, `archivable` = CASE `url_id` WHEN '31567734' THEN '1' WHEN '29358967' THEN '1' WHEN '17860022' THEN '1' WHEN '17860023' THEN '1' WHEN '34672487' THEN '1' WHEN '24762045' THEN '1' WHEN '19312685' THEN '1' WHEN '17858911' THEN '1' WHEN '27383264' THEN '1' WHEN '24763990' THEN '1' WHEN '17860024' THEN '1' WHEN '17858907' THEN '1' WHEN '27383037' THEN '1' WHEN '34672488' THEN '1' WHEN '17858939' THEN '1' WHEN '27382945' THEN '1' WHEN '27382948' THEN '1' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '1' WHEN '17859950' THEN '1' WHEN '27382914' THEN '1' WHEN '27382946' THEN '1' WHEN '17858946' THEN '1' WHEN '17858948' THEN '1' WHEN '17858948' THEN '1' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '1' WHEN '17860026' THEN '1' WHEN '17858989' THEN '1' WHEN '17858994' THEN '1' WHEN '27383500' THEN '1' WHEN '27383085' THEN '1' WHEN '27383738' THEN '1' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '1' WHEN '17859017' THEN '1' WHEN '28558712' THEN '1' WHEN '17859029' THEN '1' WHEN '23258189' THEN '1' WHEN '28558711' THEN '1' WHEN '30427325' THEN '1' WHEN '27419864' THEN '1' WHEN '19087552' THEN '1' WHEN '17859036' THEN '1' WHEN '33750214' THEN '1' WHEN '27383447' THEN '1' WHEN '27383830' THEN '1' WHEN '28574522' THEN '1' WHEN '17860027' THEN '1' WHEN '17859043' THEN '1' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '1' WHEN '17859951' THEN '1' WHEN '17860028' THEN '1' WHEN '25866129' THEN '1' WHEN '27383509' THEN '1' WHEN '27419869' THEN '1' WHEN '27276338' THEN '1' WHEN '6307332' THEN '1' WHEN '38883405' THEN '1' WHEN '36243487' THEN '1' WHEN '27315872' THEN '1' WHEN '17860029' THEN '1' WHEN '17859047' THEN '1' WHEN '27383131' THEN '1' WHEN '27383696' THEN '1' WHEN '17859952' THEN '1' WHEN '27383293' THEN '1' WHEN '27383200' THEN '1' WHEN '27383199' THEN '1' WHEN '27382961' THEN '1' END, `archived` = CASE `url_id` WHEN '31567734' THEN '2' WHEN '29358967' THEN '2' WHEN '17860022' THEN '1' WHEN '17860023' THEN '2' WHEN '34672487' THEN '2' WHEN '24762045' THEN '2' WHEN '19312685' THEN '2' WHEN '17858911' THEN '2' WHEN '27383264' THEN '1' WHEN '24763990' THEN '2' WHEN '17860024' THEN '1' WHEN '17858907' THEN '2' WHEN '27383037' THEN '1' WHEN '34672488' THEN '2' WHEN '17858939' THEN '1' WHEN '27382945' THEN '2' WHEN '27382948' THEN '2' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '2' WHEN '17859950' THEN '1' WHEN '27382914' THEN '2' WHEN '27382946' THEN '2' WHEN '17858946' THEN '2' WHEN '17858948' THEN '2' WHEN '17858948' THEN '2' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '2' WHEN '17860026' THEN '1' WHEN '17858989' THEN '2' WHEN '17858994' THEN '2' WHEN '27383500' THEN '2' WHEN '27383085' THEN '2' WHEN '27383738' THEN '2' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '2' WHEN '17859017' THEN '2' WHEN '28558712' THEN '2' WHEN '17859029' THEN '1' WHEN '23258189' THEN '2' WHEN '28558711' THEN '1' WHEN '30427325' THEN '2' WHEN '27419864' THEN '1' WHEN '19087552' THEN '2' WHEN '17859036' THEN '1' WHEN '33750214' THEN '2' WHEN '27383447' THEN '2' WHEN '27383830' THEN '2' WHEN '28574522' THEN '2' WHEN '17860027' THEN '1' WHEN '17859043' THEN '2' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '2' WHEN '17859951' THEN '2' WHEN '17860028' THEN '1' WHEN '25866129' THEN '2' WHEN '27383509' THEN '2' WHEN '27419869' THEN '1' WHEN '27276338' THEN '2' WHEN '6307332' THEN '2' WHEN '38883405' THEN '2' WHEN '36243487' THEN '2' WHEN '27315872' THEN '2' WHEN '17860029' THEN '1' WHEN '17859047' THEN '2' WHEN '27383131' THEN '2' WHEN '27383696' THEN '2' WHEN '17859952' THEN '1' WHEN '27383293' THEN '2' WHEN '27383200' THEN '2' WHEN '27383199' THEN '2' WHEN '27382961' THEN '2' END, `archive_failure` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN NULL WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN NULL WHEN '24763990' THEN NULL WHEN '17860024' THEN NULL WHEN '17858907' THEN NULL WHEN '27383037' THEN NULL WHEN '34672488' THEN NULL WHEN '17858939' THEN NULL WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN NULL WHEN '17858936' THEN NULL WHEN '24766541' THEN NULL WHEN '17859950' THEN NULL WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN NULL WHEN '30732610' THEN NULL WHEN '17859014' THEN NULL WHEN '17859015' THEN NULL WHEN '17860026' THEN NULL WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN NULL WHEN '17859022' THEN NULL WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN NULL WHEN '23258189' THEN NULL WHEN '28558711' THEN NULL WHEN '30427325' THEN NULL WHEN '27419864' THEN NULL WHEN '19087552' THEN NULL WHEN '17859036' THEN NULL WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN NULL WHEN '17859043' THEN NULL WHEN '27419867' THEN NULL WHEN '17859042' THEN NULL WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN NULL WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN NULL WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN NULL WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN NULL WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `access_time` = CASE `url_id` WHEN '31567734' THEN '2012-12-20 00:00:00' WHEN '29358967' THEN '2012-12-20 00:00:00' WHEN '17860022' THEN '2012-12-20 00:00:00' WHEN '17860023' THEN '2012-12-20 00:00:00' WHEN '34672487' THEN '2012-12-20 00:00:00' WHEN '24762045' THEN '2012-12-20 00:00:00' WHEN '19312685' THEN '2012-12-20 00:00:00' WHEN '17858911' THEN '2012-12-20 00:00:00' WHEN '27383264' THEN '2012-12-20 00:00:00' WHEN '24763990' THEN '2012-12-20 00:00:00' WHEN '17860024' THEN '2012-12-20 00:00:00' WHEN '17858907' THEN '2012-12-20 00:00:00' WHEN '27383037' THEN '2012-12-20 00:00:00' WHEN '34672488' THEN '2012-12-20 00:00:00' WHEN '17858939' THEN '2012-12-20 00:00:00' WHEN '27382945' THEN '2012-12-20 00:00:00' WHEN '27382948' THEN '2012-12-20 00:00:00' WHEN '17860025' THEN '2012-12-20 00:00:00' WHEN '17858936' THEN '2012-12-20 00:00:00' WHEN '24766541' THEN '2012-12-20 00:00:00' WHEN '17859950' THEN '2012-12-20 00:00:00' WHEN '27382914' THEN '2012-12-20 00:00:00' WHEN '27382946' THEN '2012-12-20 00:00:00' WHEN '17858946' THEN '2012-12-20 00:00:00' WHEN '17858948' THEN '2012-12-20 00:00:00' WHEN '17858948' THEN '2012-12-20 00:00:00' WHEN '17858963' THEN '2012-12-20 00:00:00' WHEN '30732610' THEN '2012-12-20 00:00:00' WHEN '17859014' THEN '2012-12-20 00:00:00' WHEN '17859015' THEN '2012-12-20 00:00:00' WHEN '17860026' THEN '2012-12-20 00:00:00' WHEN '17858989' THEN '2012-12-20 00:00:00' WHEN '17858994' THEN '2012-12-20 00:00:00' WHEN '27383500' THEN '2012-12-20 00:00:00' WHEN '27383085' THEN '2012-12-20 00:00:00' WHEN '27383738' THEN '2012-12-20 00:00:00' WHEN '17859025' THEN '2012-12-20 00:00:00' WHEN '17859022' THEN '2012-12-20 00:00:00' WHEN '17859027' THEN '2012-12-20 00:00:00' WHEN '17859017' THEN '2012-12-20 00:00:00' WHEN '28558712' THEN '2012-12-20 00:00:00' WHEN '17859029' THEN '2012-12-20 00:00:00' WHEN '23258189' THEN '2012-12-20 00:00:00' WHEN '28558711' THEN '2012-12-20 00:00:00' WHEN '30427325' THEN '2012-12-20 00:00:00' WHEN '27419864' THEN '2012-12-20 00:00:00' WHEN '19087552' THEN '2012-12-20 00:00:00' WHEN '17859036' THEN '2012-12-20 00:00:00' WHEN '33750214' THEN '2012-12-20 00:00:00' WHEN '27383447' THEN '2012-12-20 00:00:00' WHEN '27383830' THEN '2012-12-20 00:00:00' WHEN '28574522' THEN '2012-12-20 00:00:00' WHEN '17860027' THEN '2012-12-20 00:00:00' WHEN '17859043' THEN '2012-12-20 00:00:00' WHEN '27419867' THEN '2012-12-20 00:00:00' WHEN '17859042' THEN '2012-12-20 00:00:00' WHEN '19087566' THEN '2012-12-20 00:00:00' WHEN '17859951' THEN '2012-12-20 00:00:00' WHEN '17860028' THEN '2012-12-20 00:00:00' WHEN '25866129' THEN '2012-12-20 00:00:00' WHEN '27383509' THEN '2012-12-20 00:00:00' WHEN '27419869' THEN '2012-12-20 00:00:00' WHEN '27276338' THEN '2012-12-20 00:00:00' WHEN '6307332' THEN '2012-12-20 00:00:00' WHEN '38883405' THEN '2012-12-20 00:00:00' WHEN '36243487' THEN '2012-12-20 00:00:00' WHEN '27315872' THEN '2012-12-20 00:00:00' WHEN '17860029' THEN '2012-12-20 00:00:00' WHEN '17859047' THEN '2012-12-20 00:00:00' WHEN '27383131' THEN '2012-12-20 00:00:00' WHEN '27383696' THEN '2012-12-20 00:00:00' WHEN '17859952' THEN '2012-12-20 00:00:00' WHEN '27383293' THEN '2012-12-20 00:00:00' WHEN '27383200' THEN '2012-12-20 00:00:00' WHEN '27383199' THEN '2012-12-20 00:00:00' WHEN '27382961' THEN '2012-12-20 00:00:00' END, `archive_time` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN '2012-10-04 18:50:41' WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN '2016-01-06 22:53:02' WHEN '24763990' THEN NULL WHEN '17860024' THEN '2012-04-18 03:31:36' WHEN '17858907' THEN NULL WHEN '27383037' THEN '2016-05-30 00:04:07' WHEN '34672488' THEN NULL WHEN '17858939' THEN '2012-10-05 19:35:44' WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN '2012-04-17 03:04:18' WHEN '17858936' THEN '2012-10-05 19:29:28' WHEN '24766541' THEN NULL WHEN '17859950' THEN '2012-04-17 22:30:12' WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN '2016-04-26 10:22:20' WHEN '30732610' THEN '2012-04-17 09:58:10' WHEN '17859014' THEN '2011-08-11 09:07:00' WHEN '17859015' THEN NULL WHEN '17860026' THEN '2012-04-17 11:48:32' WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN '2012-10-05 19:49:55' WHEN '17859022' THEN '2012-10-05 19:53:04' WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN '2012-10-03 23:15:08' WHEN '23258189' THEN NULL WHEN '28558711' THEN '2012-04-17 06:51:16' WHEN '30427325' THEN NULL WHEN '27419864' THEN '2013-06-27 01:20:56' WHEN '19087552' THEN NULL WHEN '17859036' THEN '2011-06-26 21:55:31' WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN '2012-09-05 14:08:08' WHEN '17859043' THEN NULL WHEN '27419867' THEN '2013-09-14 21:01:43' WHEN '17859042' THEN '2013-09-12 13:02:10' WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN '2015-03-18 06:24:45' WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN '2013-06-26 16:23:37' WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN '2013-04-23 04:34:47' WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN '2013-04-25 22:04:12' WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `reviewed` = CASE `url_id` WHEN '31567734' THEN '0' WHEN '29358967' THEN '0' WHEN '17860022' THEN '0' WHEN '17860023' THEN '0' WHEN '34672487' THEN '0' WHEN '24762045' THEN '0' WHEN '19312685' THEN '0' WHEN '17858911' THEN '0' WHEN '27383264' THEN '0' WHEN '24763990' THEN '0' WHEN '17860024' THEN '0' WHEN '17858907' THEN '0' WHEN '27383037' THEN '0' WHEN '34672488' THEN '0' WHEN '17858939' THEN '0' WHEN '27382945' THEN '0' WHEN '27382948' THEN '0' WHEN '17860025' THEN '0' WHEN '17858936' THEN '0' WHEN '24766541' THEN '0' WHEN '17859950' THEN '0' WHEN '27382914' THEN '0' WHEN '27382946' THEN '0' WHEN '17858946' THEN '0' WHEN '17858948' THEN '0' WHEN '17858948' THEN '0' WHEN '17858963' THEN '0' WHEN '30732610' THEN '0' WHEN '17859014' THEN '0' WHEN '17859015' THEN '0' WHEN '17860026' THEN '0' WHEN '17858989' THEN '0' WHEN '17858994' THEN '0' WHEN '27383500' THEN '0' WHEN '27383085' THEN '0' WHEN '27383738' THEN '0' WHEN '17859025' THEN '0' WHEN '17859022' THEN '0' WHEN '17859027' THEN '0' WHEN '17859017' THEN '0' WHEN '28558712' THEN '0' WHEN '17859029' THEN '0' WHEN '23258189' THEN '0' WHEN '28558711' THEN '0' WHEN '30427325' THEN '0' WHEN '27419864' THEN '0' WHEN '19087552' THEN '0' WHEN '17859036' THEN '0' WHEN '33750214' THEN '0' WHEN '27383447' THEN '0' WHEN '27383830' THEN '0' WHEN '28574522' THEN '0' WHEN '17860027' THEN '0' WHEN '17859043' THEN '0' WHEN '27419867' THEN '0' WHEN '17859042' THEN '0' WHEN '19087566' THEN '0' WHEN '17859951' THEN '0' WHEN '17860028' THEN '0' WHEN '25866129' THEN '0' WHEN '27383509' THEN '0' WHEN '27419869' THEN '0' WHEN '27276338' THEN '0' WHEN '6307332' THEN '0' WHEN '38883405' THEN '0' WHEN '36243487' THEN '0' WHEN '27315872' THEN '0' WHEN '17860029' THEN '0' WHEN '17859047' THEN '0' WHEN '27383131' THEN '0' WHEN '27383696' THEN '0' WHEN '17859952' THEN '0' WHEN '27383293' THEN '0' WHEN '27383200' THEN '0' WHEN '27383199' THEN '0' WHEN '27382961' THEN '0' END WHERE `url_id` IN ('31567734', '29358967', '17860022', '17860023', '34672487', '24762045', '19312685', '17858911', '27383264', '24763990', '17860024', '17858907', '27383037', '34672488', '17858939', '27382945', '27382948', '17860025', '17858936', '24766541', '17859950', '27382914', '27382946', '17858946', '17858948', '17858963', '30732610', '17859014', '17859015', '17860026', '17858989', '17858994', '27383500', '27383085', '27383738', '17859025', '17859022', '17859027', '17859017', '28558712', '17859029', '23258189', '28558711', '30427325', '27419864', '19087552', '17859036', '33750214', '27383447', '27383830', '28574522', '17860027', '17859043', '27419867', '17859042', '19087566', '17859951', '17860028', '25866129', '27383509', '27419869', '27276338', '6307332', '38883405', '36243487', '27315872', '17860029', '17859047', '27383131', '27383696', '17859952', '27383293', '27383200', '27383199', '27382961') /* 7604b4b04cd2c04c7d20aa576a53f407 labsdb1005 s51059__cyberbot 4s */Comment Actions
We were commenting this morning the binary_log usage:
SELECT * FROM user_statistics ORDER BY BINLOG_BYTES_WRITTEN DESC LIMIT 10; +--------+-------------------+------------------------+----------------+--------------------+--------------------+----------------+------------+----------------------+-------------+-----------+--------------+---------------+--------------+-----------------+-----------------+----------------+---------------------+-----------------------+--------------------+------------------+---------------+---------------+ | USER | TOTAL_CONNECTIONS | CONCURRENT_CONNECTIONS | CONNECTED_TIME | BUSY_TIME | CPU_TIME | BYTES_RECEIVED | BYTES_SENT | BINLOG_BYTES_WRITTEN | ROWS_READ | ROWS_SENT | ROWS_DELETED | ROWS_INSERTED | ROWS_UPDATED | SELECT_COMMANDS | UPDATE_COMMANDS | OTHER_COMMANDS | COMMIT_TRANSACTIONS | ROLLBACK_TRANSACTIONS | DENIED_CONNECTIONS | LOST_CONNECTIONS | ACCESS_DENIED | EMPTY_QUERIES | +--------+-------------------+------------------------+----------------+--------------------+--------------------+----------------+------------+----------------------+-------------+-----------+--------------+---------------+--------------+-----------------+-----------------+----------------+---------------------+-----------------------+--------------------+------------------+---------------+---------------+ | s51059 | 3286433 | 0 | 1591624 | 49871.42916261182 | 2287.176372505849 | 8099482100 | 3079601564 | 9223372036854775807 | 463828557 | 2591788 | 18864 | 403191 | 241217 | 2071816 | 8413272 | 1641414 | 2490700 | 670 | 149 | 0 | 2 | 1098076 | | s51051 | 12541 | 0 | 82671 | 7274.119232970471 | 4899.212437269994 | 6675201633 | 372661937 | 7643997085 | 32175024 | 180848 | 31994176 | 31833015 | 0 | 1956 | 31853030 | 22548 | 63728030 | 0 | 0 | 0 | 0 | 28 | | s51412 | 45317 | 0 | 137257 | 52540.22327300166 | 5990.589198002311 | 2304676854 | 3489307753 | 2184671508 | 1783617914 | 57244183 | 1158332 | 5945482 | 5920277 | 6693432 | 4264740 | 76800 | 15188509 | 477 | 0 | 0 | 0 | 454142 | | s51138 | 4600 | 0 | 39535 | 4288.522930000422 | 1872.0174412995996 | 1121493871 | 2126767617 | 1968968166 | 271555576 | 10474072 | 1287649 | 5277385 | 102406 | 6612 | 2897595 | 5146 | 4494833 | 0 | 0 | 0 | 0 | 755 | | s52573 | 4 | 0 | 4138 | 1823.2678909939073 | 163.0070371999929 | 854098963 | 1353783172 | 1201812959 | 2449650 | 582665 | 192053 | 196655 | 117806 | 78658 | 121547 | 359965 | 480216 | 0 | 0 | 0 | 0 | 1770 | | s51211 | 12880 | 0 | 45794 | 24532.715825995576 | 17661.813545697383 | 480345467 | 519321245 | 706795833 | 16901307170 | 6122262 | 860 | 4449750 | 2890648 | 13091 | 4450852 | 3 | 4452105 | 0 | 0 | 0 | 0 | 504 | | s53220 | 220789 | 0 | 177191 | 34666.345368108574 | 32474.256873908802 | 126347735 | 200938795 | 298269817 | 14691456663 | 528025 | 0 | 73180 | 283078 | 440897 | 356259 | 220793 | 1139654 | 0 | 0 | 0 | 0 | 3063 | | s51230 | 16 | 0 | 334981 | 68880.91065299555 | 3830.047525400047 | 287240993 | 204953122 | 236995894 | 3190803326 | 387536 | 0 | 514319 | 53160 | 682343 | 567477 | 0 | 1815211 | 873 | 0 | 0 | 0 | 329536 | | s52481 | 21771 | 0 | 67733 | 9055.560536997295 | 7777.442690200963 | 187595719 | 725295797 | 172713405 | 3848335171 | 4761003 | 8412 | 635116 | 0 | 36893 | 829619 | 199049 | 1705622 | 0 | 0 | 0 | 0 | 6414 | | s52532 | 35647 | 0 | 188844 | 5766.93312300146 | 5084.554849399957 | 34178686 | 29055205 | 104911907 | 1003079476 | 198220 | 2 | 28410 | 141460 | 35445 | 85323 | 35641 | 205754 | 0 | 0 | 0 | 0 | 7061 | +--------+-------------------+------------------------+----------------+--------------------+--------------------+----------------+------------+----------------------+-------------+-----------+--------------+---------------+--------------+-----------------+-----------------+----------------+---------------------+-----------------------+--------------------+------------------+---------------+---------------+ 10 rows in set (0.00 sec)
Comment Actions
Hmm...That was the major table I spent the most time optimizing. It's pretty well indexed. That query uses the URL ID in the WHERE clause and that is the PRIMARY KEY. I don't understand how that would take long to execute. Last time I tested that giant query, it executed in a fraction of a second. How long did that take execute?
Comment Actions
That particular one took 4 seconds. (That's the "4s" in the comment at the end of the paste from Tendril.) The recent slow query log has a few other similar queries with fewer parallel updates which are showing with a 1 second runtime. There are a wide variety of things that can delay an update including maintaining indexes and waiting to acquire write locks from other threads.
Comment Actions
Unfortunately, as far as optimizations go, that query and the table are as best optimized as I know how. So yes, we should move the bot's DB to it's own VM.
Comment Actions
The graph looks stable and back to the normal pattern: https://grafana.wikimedia.org/dashboard/file/server-board.json?refresh=1m&panelId=19&fullscreen&orgId=1&var-server=labsdb1005&var-network=bond0&from=1507622609223&to=1508227409223
This bot might not be the issue, but if we see issues arising again, we will limit the bot back to 10-20 connections and see what happens (we will reopen this ticket too so you know that it can fail again).
Let's continue on: T178332
Going to close this ticket as its scope is done - the bot not working works since the connection limit was removed.