Cyberpower678 (Maximilian Doerr)
Voluntary Software Engineer/Bot Operator

Projects (8)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Friday

  • Clear sailing ahead.

User Details

User Since
Nov 14 2014, 4:15 PM (152 w, 5 d)
Availability
Available
IRC Nick
Cyberpower678 or CP678
LDAP User
Cyberpower678
MediaWiki User
Cyberpower678

Recent Activity

Today

Cyberpower678 added a comment to T178332: Request increased quota for cyberbot Cloud VPS project.

I have now spawned a large instance named "cyberbot-db-01". Thank you for the increase.

Wed, Oct 18, 2:10 PM · Cloud-VPS (Quota-requests)
Cyberpower678 closed T176579: Investigate supporting MMS protocol on the checkIfDead class as Resolved.
Wed, Oct 18, 2:31 AM · InternetArchiveBot (v1.6)

Mon, Oct 16

Cyberpower678 created T178332: Request increased quota for cyberbot Cloud VPS project.
Mon, Oct 16, 7:57 PM · Cloud-VPS (Quota-requests)
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Hmm...That was the major table I spent the most time optimizing. It's pretty well indexed. That query uses the URL ID in the WHERE clause and that is the PRIMARY KEY. I don't understand how that would take long to execute. Last time I tested that giant query, it executed in a fraction of a second. How long did that take execute?

That particular one took 4 seconds. (That's the "4s" in the comment at the end of the paste from Tendril.) The recent slow query log has a few other similar queries with fewer parallel updates which are showing with a 1 second runtime. There are a wide variety of things that can delay an update including maintaining indexes and waiting to acquire write locks from other threads.

Mon, Oct 16, 7:17 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

tools.cyberbot (s51059) is currently the number 5 user of disk space on tools.labsdb (49.6 GB). The externallinks_global table has grown to 36G which may have changed the performance footprint from the last time that we dug into this tool's database usage. More tools are migrating their data to tools.labsdb as well due to the impending shutdown of c1.labsdb and c3.labsdb which have historically been used by some tools for persistent data storage. This is a round about way of saying that these are dynamic systems and what worked yesterday may or may not work today or tomorrow.


Here's an example of a slow query as reported by tendril:

UPDATE `externallinks_global` SET `archive_url` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN 'https://web.archive.org/web/20121004185041/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2001.pdf' WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN 'https://web.archive.org/web/20160106225302/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2005/405.pdf' WHEN '24763990' THEN NULL WHEN '17860024' THEN 'https://web.archive.org/web/20120418033136/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2005.pdf' WHEN '17858907' THEN NULL WHEN '27383037' THEN 'https://web.archive.org/web/20160530000407/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2005/1029.pdf' WHEN '34672488' THEN NULL WHEN '17858939' THEN 'https://web.archive.org/web/20121005193544/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2006/904.pdf' WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN 'https://web.archive.org/web/20120417030418/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2006.pdf' WHEN '17858936' THEN 'https://web.archive.org/web/20121005192928/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2006/901.pdf' WHEN '24766541' THEN NULL WHEN '17859950' THEN 'https://web.archive.org/web/20120417223012/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Doubles_Numeric_2006.pdf' WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN 'https://web.archive.org/web/20160426102220/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2007/408.pdf' WHEN '30732610' THEN 'https://web.archive.org/web/20120417095810/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2007.pdf' WHEN '17859014' THEN 'https://web.archive.org/web/20110811090700/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2008/904.pdf' WHEN '17859015' THEN NULL WHEN '17860026' THEN 'https://web.archive.org/web/20120417114832/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2008.pdf' WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN 'https://web.archive.org/web/20121005194955/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/901.pdf' WHEN '17859022' THEN 'https://web.archive.org/web/20121005195304/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/709.pdf' WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN 'https://web.archive.org/web/20121003231508/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2009/905.pdf' WHEN '23258189' THEN NULL WHEN '28558711' THEN 'https://web.archive.org/web/20120417065116/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2009.pdf' WHEN '30427325' THEN NULL WHEN '27419864' THEN 'https://web.archive.org/web/20130627012056/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2010/904.pdf' WHEN '19087552' THEN NULL WHEN '17859036' THEN 'https://web.archive.org/web/20110626215531/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2010/905.pdf' WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN 'https://web.archive.org/web/20120905140808/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2010.pdf' WHEN '17859043' THEN NULL WHEN '27419867' THEN 'https://web.archive.org/web/20130914210143/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2011/904.pdf' WHEN '17859042' THEN 'https://web.archive.org/web/20130912130210/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2011/901.pdf' WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN 'https://web.archive.org/web/20150318062445/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2011.pdf' WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN 'https://web.archive.org/web/20130626162337/http://www.wtatennis.com/SEWTATour-Archive/Archive/Draws/2012/904.pdf' WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN 'https://web.archive.org/web/20130423043447/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Singles_Numeric_2012.pdf' WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN 'https://web.archive.org/web/20130425220412/http://www.wtatennis.com/SEWTATour-Archive/Rankings_Stats/Doubles_Numeric_2012.pdf' WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `has_archive` = CASE `url_id` WHEN '31567734' THEN '0' WHEN '29358967' THEN '0' WHEN '17860022' THEN '1' WHEN '17860023' THEN '0' WHEN '34672487' THEN '0' WHEN '24762045' THEN '0' WHEN '19312685' THEN '0' WHEN '17858911' THEN '0' WHEN '27383264' THEN '1' WHEN '24763990' THEN '0' WHEN '17860024' THEN '1' WHEN '17858907' THEN '0' WHEN '27383037' THEN '1' WHEN '34672488' THEN '0' WHEN '17858939' THEN '1' WHEN '27382945' THEN '0' WHEN '27382948' THEN '0' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '0' WHEN '17859950' THEN '1' WHEN '27382914' THEN '0' WHEN '27382946' THEN '0' WHEN '17858946' THEN '0' WHEN '17858948' THEN '0' WHEN '17858948' THEN '0' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '0' WHEN '17860026' THEN '1' WHEN '17858989' THEN '0' WHEN '17858994' THEN '0' WHEN '27383500' THEN '0' WHEN '27383085' THEN '0' WHEN '27383738' THEN '0' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '0' WHEN '17859017' THEN '0' WHEN '28558712' THEN '0' WHEN '17859029' THEN '1' WHEN '23258189' THEN '0' WHEN '28558711' THEN '1' WHEN '30427325' THEN '0' WHEN '27419864' THEN '1' WHEN '19087552' THEN '0' WHEN '17859036' THEN '1' WHEN '33750214' THEN '0' WHEN '27383447' THEN '0' WHEN '27383830' THEN '0' WHEN '28574522' THEN '0' WHEN '17860027' THEN '1' WHEN '17859043' THEN '0' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '0' WHEN '17859951' THEN '0' WHEN '17860028' THEN '1' WHEN '25866129' THEN '0' WHEN '27383509' THEN '0' WHEN '27419869' THEN '1' WHEN '27276338' THEN '0' WHEN '6307332' THEN '0' WHEN '38883405' THEN '0' WHEN '36243487' THEN '0' WHEN '27315872' THEN '0' WHEN '17860029' THEN '1' WHEN '17859047' THEN '0' WHEN '27383131' THEN '0' WHEN '27383696' THEN '0' WHEN '17859952' THEN '1' WHEN '27383293' THEN '0' WHEN '27383200' THEN '0' WHEN '27383199' THEN '0' WHEN '27382961' THEN '0' END, `live_state` = CASE `url_id` WHEN '31567734' THEN '1' WHEN '29358967' THEN '1' WHEN '17860022' THEN '0' WHEN '17860023' THEN '1' WHEN '34672487' THEN '1' WHEN '24762045' THEN '1' WHEN '19312685' THEN '1' WHEN '17858911' THEN '5' WHEN '27383264' THEN '5' WHEN '24763990' THEN '5' WHEN '17860024' THEN '0' WHEN '17858907' THEN '5' WHEN '27383037' THEN '5' WHEN '34672488' THEN '1' WHEN '17858939' THEN '0' WHEN '27382945' THEN '5' WHEN '27382948' THEN '5' WHEN '17860025' THEN '0' WHEN '17858936' THEN '0' WHEN '24766541' THEN '1' WHEN '17859950' THEN '0' WHEN '27382914' THEN '5' WHEN '27382946' THEN '5' WHEN '17858946' THEN '5' WHEN '17858948' THEN '5' WHEN '17858948' THEN '5' WHEN '17858963' THEN '5' WHEN '30732610' THEN '0' WHEN '17859014' THEN '0' WHEN '17859015' THEN '5' WHEN '17860026' THEN '0' WHEN '17858989' THEN '5' WHEN '17858994' THEN '5' WHEN '27383500' THEN '5' WHEN '27383085' THEN '5' WHEN '27383738' THEN '5' WHEN '17859025' THEN '5' WHEN '17859022' THEN '5' WHEN '17859027' THEN '5' WHEN '17859017' THEN '5' WHEN '28558712' THEN '1' WHEN '17859029' THEN '5' WHEN '23258189' THEN '5' WHEN '28558711' THEN '0' WHEN '30427325' THEN '1' WHEN '27419864' THEN '0' WHEN '19087552' THEN '5' WHEN '17859036' THEN '5' WHEN '33750214' THEN '1' WHEN '27383447' THEN '5' WHEN '27383830' THEN '5' WHEN '28574522' THEN '1' WHEN '17860027' THEN '0' WHEN '17859043' THEN '5' WHEN '27419867' THEN '0' WHEN '17859042' THEN '0' WHEN '19087566' THEN '5' WHEN '17859951' THEN '1' WHEN '17860028' THEN '0' WHEN '25866129' THEN '5' WHEN '27383509' THEN '5' WHEN '27419869' THEN '0' WHEN '27276338' THEN '5' WHEN '6307332' THEN '5' WHEN '38883405' THEN '3' WHEN '36243487' THEN '3' WHEN '27315872' THEN '5' WHEN '17860029' THEN '0' WHEN '17859047' THEN '5' WHEN '27383131' THEN '5' WHEN '27383696' THEN '5' WHEN '17859952' THEN '0' WHEN '27383293' THEN '5' WHEN '27383200' THEN '5' WHEN '27383199' THEN '5' WHEN '27382961' THEN '5' END, `last_deadCheck` = CASE `url_id` WHEN '31567734' THEN '2017-10-16 17:37:00' WHEN '29358967' THEN '2017-10-16 17:37:00' WHEN '17860022' THEN '2017-10-16 17:37:00' WHEN '17860023' THEN '2017-10-16 17:37:00' WHEN '34672487' THEN '2017-10-16 17:37:00' WHEN '24762045' THEN '2017-10-16 17:37:00' WHEN '19312685' THEN '2017-10-16 17:37:00' WHEN '17858911' THEN '2017-06-17 14:09:06' WHEN '27383264' THEN '0000-00-00 00:00:00' WHEN '24763990' THEN '2017-06-17 13:43:19' WHEN '17860024' THEN '2017-10-16 17:37:00' WHEN '17858907' THEN '2017-06-17 12:43:38' WHEN '27383037' THEN '0000-00-00 00:00:00' WHEN '34672488' THEN '2017-10-16 17:37:00' WHEN '17858939' THEN '2017-06-17 18:27:09' WHEN '27382945' THEN '0000-00-00 00:00:00' WHEN '27382948' THEN '0000-00-00 00:00:00' WHEN '17860025' THEN '2017-09-23 03:05:09' WHEN '17858936' THEN '2017-06-08 09:20:00' WHEN '24766541' THEN '2017-10-16 17:37:00' WHEN '17859950' THEN '2017-10-16 17:37:00' WHEN '27382914' THEN '0000-00-00 00:00:00' WHEN '27382946' THEN '0000-00-00 00:00:00' WHEN '17858946' THEN '2017-06-17 21:00:24' WHEN '17858948' THEN '2017-06-17 21:39:43' WHEN '17858948' THEN '2017-06-17 21:39:43' WHEN '17858963' THEN '2017-06-17 20:24:14' WHEN '30732610' THEN '2017-10-16 17:37:00' WHEN '17859014' THEN '2017-06-18 06:20:48' WHEN '17859015' THEN '2017-06-18 06:08:16' WHEN '17860026' THEN '2017-10-16 17:37:00' WHEN '17858989' THEN '2017-06-18 02:52:40' WHEN '17858994' THEN '2017-06-18 03:35:39' WHEN '27383500' THEN '0000-00-00 00:00:00' WHEN '27383085' THEN '0000-00-00 00:00:00' WHEN '27383738' THEN '0000-00-00 00:00:00' WHEN '17859025' THEN '2017-06-18 10:06:01' WHEN '17859022' THEN '2017-06-18 12:00:06' WHEN '17859027' THEN '2017-06-08 09:20:00' WHEN '17859017' THEN '2017-06-08 09:20:00' WHEN '28558712' THEN '2017-10-16 17:37:00' WHEN '17859029' THEN '2017-06-18 14:39:25' WHEN '23258189' THEN '2017-05-15 02:31:30' WHEN '28558711' THEN '2017-09-23 03:05:09' WHEN '30427325' THEN '2017-10-16 17:37:00' WHEN '27419864' THEN '2017-10-05 16:26:26' WHEN '19087552' THEN '2017-06-08 09:20:00' WHEN '17859036' THEN '2017-06-19 00:28:15' WHEN '33750214' THEN '2017-10-16 17:37:00' WHEN '27383447' THEN '0000-00-00 00:00:00' WHEN '27383830' THEN '0000-00-00 00:00:00' WHEN '28574522' THEN '2017-10-16 17:37:00' WHEN '17860027' THEN '2017-09-22 21:26:57' WHEN '17859043' THEN '2017-06-19 07:31:25' WHEN '27419867' THEN '2017-10-05 16:26:31' WHEN '17859042' THEN '2017-10-16 17:37:00' WHEN '19087566' THEN '2017-06-08 09:20:00' WHEN '17859951' THEN '2017-10-16 17:37:00' WHEN '17860028' THEN '2017-10-11 04:21:25' WHEN '25866129' THEN '0000-00-00 00:00:00' WHEN '27383509' THEN '0000-00-00 00:00:00' WHEN '27419869' THEN '2017-10-05 16:26:36' WHEN '27276338' THEN '0000-00-00 00:00:00' WHEN '6307332' THEN '0000-00-00 00:00:00' WHEN '38883405' THEN '2017-10-15 02:47:31' WHEN '36243487' THEN '2017-10-16 17:37:00' WHEN '27315872' THEN '0000-00-00 00:00:00' WHEN '17860029' THEN '2017-10-16 17:37:00' WHEN '17859047' THEN '2016-11-05 12:25:00' WHEN '27383131' THEN '0000-00-00 00:00:00' WHEN '27383696' THEN '0000-00-00 00:00:00' WHEN '17859952' THEN '2017-10-16 17:37:00' WHEN '27383293' THEN '0000-00-00 00:00:00' WHEN '27383200' THEN '0000-00-00 00:00:00' WHEN '27383199' THEN '0000-00-00 00:00:00' WHEN '27382961' THEN '0000-00-00 00:00:00' END, `archivable` = CASE `url_id` WHEN '31567734' THEN '1' WHEN '29358967' THEN '1' WHEN '17860022' THEN '1' WHEN '17860023' THEN '1' WHEN '34672487' THEN '1' WHEN '24762045' THEN '1' WHEN '19312685' THEN '1' WHEN '17858911' THEN '1' WHEN '27383264' THEN '1' WHEN '24763990' THEN '1' WHEN '17860024' THEN '1' WHEN '17858907' THEN '1' WHEN '27383037' THEN '1' WHEN '34672488' THEN '1' WHEN '17858939' THEN '1' WHEN '27382945' THEN '1' WHEN '27382948' THEN '1' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '1' WHEN '17859950' THEN '1' WHEN '27382914' THEN '1' WHEN '27382946' THEN '1' WHEN '17858946' THEN '1' WHEN '17858948' THEN '1' WHEN '17858948' THEN '1' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '1' WHEN '17860026' THEN '1' WHEN '17858989' THEN '1' WHEN '17858994' THEN '1' WHEN '27383500' THEN '1' WHEN '27383085' THEN '1' WHEN '27383738' THEN '1' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '1' WHEN '17859017' THEN '1' WHEN '28558712' THEN '1' WHEN '17859029' THEN '1' WHEN '23258189' THEN '1' WHEN '28558711' THEN '1' WHEN '30427325' THEN '1' WHEN '27419864' THEN '1' WHEN '19087552' THEN '1' WHEN '17859036' THEN '1' WHEN '33750214' THEN '1' WHEN '27383447' THEN '1' WHEN '27383830' THEN '1' WHEN '28574522' THEN '1' WHEN '17860027' THEN '1' WHEN '17859043' THEN '1' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '1' WHEN '17859951' THEN '1' WHEN '17860028' THEN '1' WHEN '25866129' THEN '1' WHEN '27383509' THEN '1' WHEN '27419869' THEN '1' WHEN '27276338' THEN '1' WHEN '6307332' THEN '1' WHEN '38883405' THEN '1' WHEN '36243487' THEN '1' WHEN '27315872' THEN '1' WHEN '17860029' THEN '1' WHEN '17859047' THEN '1' WHEN '27383131' THEN '1' WHEN '27383696' THEN '1' WHEN '17859952' THEN '1' WHEN '27383293' THEN '1' WHEN '27383200' THEN '1' WHEN '27383199' THEN '1' WHEN '27382961' THEN '1' END, `archived` = CASE `url_id` WHEN '31567734' THEN '2' WHEN '29358967' THEN '2' WHEN '17860022' THEN '1' WHEN '17860023' THEN '2' WHEN '34672487' THEN '2' WHEN '24762045' THEN '2' WHEN '19312685' THEN '2' WHEN '17858911' THEN '2' WHEN '27383264' THEN '1' WHEN '24763990' THEN '2' WHEN '17860024' THEN '1' WHEN '17858907' THEN '2' WHEN '27383037' THEN '1' WHEN '34672488' THEN '2' WHEN '17858939' THEN '1' WHEN '27382945' THEN '2' WHEN '27382948' THEN '2' WHEN '17860025' THEN '1' WHEN '17858936' THEN '1' WHEN '24766541' THEN '2' WHEN '17859950' THEN '1' WHEN '27382914' THEN '2' WHEN '27382946' THEN '2' WHEN '17858946' THEN '2' WHEN '17858948' THEN '2' WHEN '17858948' THEN '2' WHEN '17858963' THEN '1' WHEN '30732610' THEN '1' WHEN '17859014' THEN '1' WHEN '17859015' THEN '2' WHEN '17860026' THEN '1' WHEN '17858989' THEN '2' WHEN '17858994' THEN '2' WHEN '27383500' THEN '2' WHEN '27383085' THEN '2' WHEN '27383738' THEN '2' WHEN '17859025' THEN '1' WHEN '17859022' THEN '1' WHEN '17859027' THEN '2' WHEN '17859017' THEN '2' WHEN '28558712' THEN '2' WHEN '17859029' THEN '1' WHEN '23258189' THEN '2' WHEN '28558711' THEN '1' WHEN '30427325' THEN '2' WHEN '27419864' THEN '1' WHEN '19087552' THEN '2' WHEN '17859036' THEN '1' WHEN '33750214' THEN '2' WHEN '27383447' THEN '2' WHEN '27383830' THEN '2' WHEN '28574522' THEN '2' WHEN '17860027' THEN '1' WHEN '17859043' THEN '2' WHEN '27419867' THEN '1' WHEN '17859042' THEN '1' WHEN '19087566' THEN '2' WHEN '17859951' THEN '2' WHEN '17860028' THEN '1' WHEN '25866129' THEN '2' WHEN '27383509' THEN '2' WHEN '27419869' THEN '1' WHEN '27276338' THEN '2' WHEN '6307332' THEN '2' WHEN '38883405' THEN '2' WHEN '36243487' THEN '2' WHEN '27315872' THEN '2' WHEN '17860029' THEN '1' WHEN '17859047' THEN '2' WHEN '27383131' THEN '2' WHEN '27383696' THEN '2' WHEN '17859952' THEN '1' WHEN '27383293' THEN '2' WHEN '27383200' THEN '2' WHEN '27383199' THEN '2' WHEN '27382961' THEN '2' END, `archive_failure` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN NULL WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN NULL WHEN '24763990' THEN NULL WHEN '17860024' THEN NULL WHEN '17858907' THEN NULL WHEN '27383037' THEN NULL WHEN '34672488' THEN NULL WHEN '17858939' THEN NULL WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN NULL WHEN '17858936' THEN NULL WHEN '24766541' THEN NULL WHEN '17859950' THEN NULL WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN NULL WHEN '30732610' THEN NULL WHEN '17859014' THEN NULL WHEN '17859015' THEN NULL WHEN '17860026' THEN NULL WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN NULL WHEN '17859022' THEN NULL WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN NULL WHEN '23258189' THEN NULL WHEN '28558711' THEN NULL WHEN '30427325' THEN NULL WHEN '27419864' THEN NULL WHEN '19087552' THEN NULL WHEN '17859036' THEN NULL WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN NULL WHEN '17859043' THEN NULL WHEN '27419867' THEN NULL WHEN '17859042' THEN NULL WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN NULL WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN NULL WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN NULL WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN NULL WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `access_time` = CASE `url_id` WHEN '31567734' THEN '2012-12-20 00:00:00' WHEN '29358967' THEN '2012-12-20 00:00:00' WHEN '17860022' THEN '2012-12-20 00:00:00' WHEN '17860023' THEN '2012-12-20 00:00:00' WHEN '34672487' THEN '2012-12-20 00:00:00' WHEN '24762045' THEN '2012-12-20 00:00:00' WHEN '19312685' THEN '2012-12-20 00:00:00' WHEN '17858911' THEN '2012-12-20 00:00:00' WHEN '27383264' THEN '2012-12-20 00:00:00' WHEN '24763990' THEN '2012-12-20 00:00:00' WHEN '17860024' THEN '2012-12-20 00:00:00' WHEN '17858907' THEN '2012-12-20 00:00:00' WHEN '27383037' THEN '2012-12-20 00:00:00' WHEN '34672488' THEN '2012-12-20 00:00:00' WHEN '17858939' THEN '2012-12-20 00:00:00' WHEN '27382945' THEN '2012-12-20 00:00:00' WHEN '27382948' THEN '2012-12-20 00:00:00' WHEN '17860025' THEN '2012-12-20 00:00:00' WHEN '17858936' THEN '2012-12-20 00:00:00' WHEN '24766541' THEN '2012-12-20 00:00:00' WHEN '17859950' THEN '2012-12-20 00:00:00' WHEN '27382914' THEN '2012-12-20 00:00:00' WHEN '27382946' THEN '2012-12-20 00:00:00' WHEN '17858946' THEN '2012-12-20 00:00:00' WHEN '17858948' THEN '2012-12-20 00:00:00' WHEN '17858948' THEN '2012-12-20 00:00:00' WHEN '17858963' THEN '2012-12-20 00:00:00' WHEN '30732610' THEN '2012-12-20 00:00:00' WHEN '17859014' THEN '2012-12-20 00:00:00' WHEN '17859015' THEN '2012-12-20 00:00:00' WHEN '17860026' THEN '2012-12-20 00:00:00' WHEN '17858989' THEN '2012-12-20 00:00:00' WHEN '17858994' THEN '2012-12-20 00:00:00' WHEN '27383500' THEN '2012-12-20 00:00:00' WHEN '27383085' THEN '2012-12-20 00:00:00' WHEN '27383738' THEN '2012-12-20 00:00:00' WHEN '17859025' THEN '2012-12-20 00:00:00' WHEN '17859022' THEN '2012-12-20 00:00:00' WHEN '17859027' THEN '2012-12-20 00:00:00' WHEN '17859017' THEN '2012-12-20 00:00:00' WHEN '28558712' THEN '2012-12-20 00:00:00' WHEN '17859029' THEN '2012-12-20 00:00:00' WHEN '23258189' THEN '2012-12-20 00:00:00' WHEN '28558711' THEN '2012-12-20 00:00:00' WHEN '30427325' THEN '2012-12-20 00:00:00' WHEN '27419864' THEN '2012-12-20 00:00:00' WHEN '19087552' THEN '2012-12-20 00:00:00' WHEN '17859036' THEN '2012-12-20 00:00:00' WHEN '33750214' THEN '2012-12-20 00:00:00' WHEN '27383447' THEN '2012-12-20 00:00:00' WHEN '27383830' THEN '2012-12-20 00:00:00' WHEN '28574522' THEN '2012-12-20 00:00:00' WHEN '17860027' THEN '2012-12-20 00:00:00' WHEN '17859043' THEN '2012-12-20 00:00:00' WHEN '27419867' THEN '2012-12-20 00:00:00' WHEN '17859042' THEN '2012-12-20 00:00:00' WHEN '19087566' THEN '2012-12-20 00:00:00' WHEN '17859951' THEN '2012-12-20 00:00:00' WHEN '17860028' THEN '2012-12-20 00:00:00' WHEN '25866129' THEN '2012-12-20 00:00:00' WHEN '27383509' THEN '2012-12-20 00:00:00' WHEN '27419869' THEN '2012-12-20 00:00:00' WHEN '27276338' THEN '2012-12-20 00:00:00' WHEN '6307332' THEN '2012-12-20 00:00:00' WHEN '38883405' THEN '2012-12-20 00:00:00' WHEN '36243487' THEN '2012-12-20 00:00:00' WHEN '27315872' THEN '2012-12-20 00:00:00' WHEN '17860029' THEN '2012-12-20 00:00:00' WHEN '17859047' THEN '2012-12-20 00:00:00' WHEN '27383131' THEN '2012-12-20 00:00:00' WHEN '27383696' THEN '2012-12-20 00:00:00' WHEN '17859952' THEN '2012-12-20 00:00:00' WHEN '27383293' THEN '2012-12-20 00:00:00' WHEN '27383200' THEN '2012-12-20 00:00:00' WHEN '27383199' THEN '2012-12-20 00:00:00' WHEN '27382961' THEN '2012-12-20 00:00:00' END, `archive_time` = CASE `url_id` WHEN '31567734' THEN NULL WHEN '29358967' THEN NULL WHEN '17860022' THEN '2012-10-04 18:50:41' WHEN '17860023' THEN NULL WHEN '34672487' THEN NULL WHEN '24762045' THEN NULL WHEN '19312685' THEN NULL WHEN '17858911' THEN NULL WHEN '27383264' THEN '2016-01-06 22:53:02' WHEN '24763990' THEN NULL WHEN '17860024' THEN '2012-04-18 03:31:36' WHEN '17858907' THEN NULL WHEN '27383037' THEN '2016-05-30 00:04:07' WHEN '34672488' THEN NULL WHEN '17858939' THEN '2012-10-05 19:35:44' WHEN '27382945' THEN NULL WHEN '27382948' THEN NULL WHEN '17860025' THEN '2012-04-17 03:04:18' WHEN '17858936' THEN '2012-10-05 19:29:28' WHEN '24766541' THEN NULL WHEN '17859950' THEN '2012-04-17 22:30:12' WHEN '27382914' THEN NULL WHEN '27382946' THEN NULL WHEN '17858946' THEN NULL WHEN '17858948' THEN NULL WHEN '17858948' THEN NULL WHEN '17858963' THEN '2016-04-26 10:22:20' WHEN '30732610' THEN '2012-04-17 09:58:10' WHEN '17859014' THEN '2011-08-11 09:07:00' WHEN '17859015' THEN NULL WHEN '17860026' THEN '2012-04-17 11:48:32' WHEN '17858989' THEN NULL WHEN '17858994' THEN NULL WHEN '27383500' THEN NULL WHEN '27383085' THEN NULL WHEN '27383738' THEN NULL WHEN '17859025' THEN '2012-10-05 19:49:55' WHEN '17859022' THEN '2012-10-05 19:53:04' WHEN '17859027' THEN NULL WHEN '17859017' THEN NULL WHEN '28558712' THEN NULL WHEN '17859029' THEN '2012-10-03 23:15:08' WHEN '23258189' THEN NULL WHEN '28558711' THEN '2012-04-17 06:51:16' WHEN '30427325' THEN NULL WHEN '27419864' THEN '2013-06-27 01:20:56' WHEN '19087552' THEN NULL WHEN '17859036' THEN '2011-06-26 21:55:31' WHEN '33750214' THEN NULL WHEN '27383447' THEN NULL WHEN '27383830' THEN NULL WHEN '28574522' THEN NULL WHEN '17860027' THEN '2012-09-05 14:08:08' WHEN '17859043' THEN NULL WHEN '27419867' THEN '2013-09-14 21:01:43' WHEN '17859042' THEN '2013-09-12 13:02:10' WHEN '19087566' THEN NULL WHEN '17859951' THEN NULL WHEN '17860028' THEN '2015-03-18 06:24:45' WHEN '25866129' THEN NULL WHEN '27383509' THEN NULL WHEN '27419869' THEN '2013-06-26 16:23:37' WHEN '27276338' THEN NULL WHEN '6307332' THEN NULL WHEN '38883405' THEN NULL WHEN '36243487' THEN NULL WHEN '27315872' THEN NULL WHEN '17860029' THEN '2013-04-23 04:34:47' WHEN '17859047' THEN NULL WHEN '27383131' THEN NULL WHEN '27383696' THEN NULL WHEN '17859952' THEN '2013-04-25 22:04:12' WHEN '27383293' THEN NULL WHEN '27383200' THEN NULL WHEN '27383199' THEN NULL WHEN '27382961' THEN NULL END, `reviewed` = CASE `url_id` WHEN '31567734' THEN '0' WHEN '29358967' THEN '0' WHEN '17860022' THEN '0' WHEN '17860023' THEN '0' WHEN '34672487' THEN '0' WHEN '24762045' THEN '0' WHEN '19312685' THEN '0' WHEN '17858911' THEN '0' WHEN '27383264' THEN '0' WHEN '24763990' THEN '0' WHEN '17860024' THEN '0' WHEN '17858907' THEN '0' WHEN '27383037' THEN '0' WHEN '34672488' THEN '0' WHEN '17858939' THEN '0' WHEN '27382945' THEN '0' WHEN '27382948' THEN '0' WHEN '17860025' THEN '0' WHEN '17858936' THEN '0' WHEN '24766541' THEN '0' WHEN '17859950' THEN '0' WHEN '27382914' THEN '0' WHEN '27382946' THEN '0' WHEN '17858946' THEN '0' WHEN '17858948' THEN '0' WHEN '17858948' THEN '0' WHEN '17858963' THEN '0' WHEN '30732610' THEN '0' WHEN '17859014' THEN '0' WHEN '17859015' THEN '0' WHEN '17860026' THEN '0' WHEN '17858989' THEN '0' WHEN '17858994' THEN '0' WHEN '27383500' THEN '0' WHEN '27383085' THEN '0' WHEN '27383738' THEN '0' WHEN '17859025' THEN '0' WHEN '17859022' THEN '0' WHEN '17859027' THEN '0' WHEN '17859017' THEN '0' WHEN '28558712' THEN '0' WHEN '17859029' THEN '0' WHEN '23258189' THEN '0' WHEN '28558711' THEN '0' WHEN '30427325' THEN '0' WHEN '27419864' THEN '0' WHEN '19087552' THEN '0' WHEN '17859036' THEN '0' WHEN '33750214' THEN '0' WHEN '27383447' THEN '0' WHEN '27383830' THEN '0' WHEN '28574522' THEN '0' WHEN '17860027' THEN '0' WHEN '17859043' THEN '0' WHEN '27419867' THEN '0' WHEN '17859042' THEN '0' WHEN '19087566' THEN '0' WHEN '17859951' THEN '0' WHEN '17860028' THEN '0' WHEN '25866129' THEN '0' WHEN '27383509' THEN '0' WHEN '27419869' THEN '0' WHEN '27276338' THEN '0' WHEN '6307332' THEN '0' WHEN '38883405' THEN '0' WHEN '36243487' THEN '0' WHEN '27315872' THEN '0' WHEN '17860029' THEN '0' WHEN '17859047' THEN '0' WHEN '27383131' THEN '0' WHEN '27383696' THEN '0' WHEN '17859952' THEN '0' WHEN '27383293' THEN '0' WHEN '27383200' THEN '0' WHEN '27383199' THEN '0' WHEN '27382961' THEN '0' END WHERE `url_id` IN ('31567734', '29358967', '17860022', '17860023', '34672487', '24762045', '19312685', '17858911', '27383264', '24763990', '17860024', '17858907', '27383037', '34672488', '17858939', '27382945', '27382948', '17860025', '17858936', '24766541', '17859950', '27382914', '27382946', '17858946', '17858948', '17858963', '30732610', '17859014', '17859015', '17860026', '17858989', '17858994', '27383500', '27383085', '27383738', '17859025', '17859022', '17859027', '17859017', '28558712', '17859029', '23258189', '28558711', '30427325', '27419864', '19087552', '17859036', '33750214', '27383447', '27383830', '28574522', '17860027', '17859043', '27419867', '17859042', '19087566', '17859951', '17860028', '25866129', '27383509', '27419869', '27276338', '6307332', '38883405', '36243487', '27315872', '17860029', '17859047', '27383131', '27383696', '17859952', '27383293', '27383200', '27383199', '27382961') /* 7604b4b04cd2c04c7d20aa576a53f407 labsdb1005 s51059__cyberbot 4s */
Mon, Oct 16, 7:07 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

I do find that somewhat difficult to believe. When the new DB was tested, that was with 28 concurrent connections and the load didn’t go up. Looking at the past 30 days, where the bot has consistently been running the same queries throughout I see the load going up and down. Something here just doesn’t line up with my line of logic. Don’t misunderstand. I’m not being confrontational, just trying to understand. IABot, even with the number of connections, should not be spiking the load that significantly. Are there any queries you can point me to that maybe I can optimize?

Mon, Oct 16, 6:07 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 updated subscribers of T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Cyberpower678 file a ticket to wmcs Data services, and CC me. Do not expect it to be quick. For the time being, you should try to stick to 10 threads- if the server crashes and stops giving service, that will be much worse than having less available connections. If the server crashes again, the limit will be reinstated indefinitely- not only to you, to all users.

Mon, Oct 16, 5:36 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

2017-10-16 17:01:25: (mod_fastcgi.c.2673) FastCGI-stderr: PHP Warning: mysqli_connect(): (HY000/1226): User 's51059' has exceeded the 'max_user_connections' resource (current value: 10) in /mnt/nfs/labstore-secondary-tools-project/iabot/public_html/Includes/DB2.php on line 27

I have now removed the limit.

Mon, Oct 16, 5:24 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

2017-10-16 17:01:25: (mod_fastcgi.c.2673) FastCGI-stderr: PHP Warning: mysqli_connect(): (HY000/1226): User 's51059' has exceeded the 'max_user_connections' resource (current value: 10) in /mnt/nfs/labstore-secondary-tools-project/iabot/public_html/Includes/DB2.php on line 27

Mon, Oct 16, 5:05 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

BTW, I'm still getting the errors.

It works for me everytime I connect to the URL you pasted earlier.

Mon, Oct 16, 4:05 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

BTW, I'm still getting the errors.

Mon, Oct 16, 4:02 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

And I thought it was harder on the server to constantly reconnect

If this was a dedicated server, pooling would be preferred. Being a highly-shared database, it is better to disconnect ASAP. The "rule" is to enforce that. Not doing that can make the server incompatible in the future- so better adapt now.

Alternative, you can consider asking for dedicated resources- it may be possible with soon to be installed new database hardware (but obviously, we cannot guarantee to be granted; but I would personally would back that request).

Mon, Oct 16, 3:51 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

It started to work after I removed the limit of 10 connections, so it was clearly trying to create more than 10 at the time. So far it is fine, but if we see that the host load starts to bump again, we might need to limit it.
This might not even be the cause of the server issue anyways and it happened to be triaged while we were trying to get the server back stable :-)

Mon, Oct 16, 3:46 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

And I thought it was harder on the server to constantly reconnect when making a query. Where are those two connections coming from?

Mon, Oct 16, 3:42 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Hmm... IABot doesn't leave connections to idle. Where are they coming from. I do have Workbench open on another machine.

Mon, Oct 16, 3:39 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Yes it is still happening. It usually happens when it can't connect to the DB.

We had to limit that user to 10 connections as we were not sure it was causing the issues with the server.
Right now those 10 connections are open and thus it is probably getting errors because of that.
Those connections are not doing anything, just sleeping.
Ideally it should not leave lots of connections opens, it should just open a connection, use it, and close it. This server is shared, so we need to limit resources,.

I have removed the limit and I can see that: https://tools.wmflabs.org/iabot/ works again.

As I said, can you please try to change it in order to open-close connections, otherwise, whenever we have to limit it (maybe not to 10, but to some other value) it will fail.

Mon, Oct 16, 3:34 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Yes it is still happening. It usually happens when it can't connect to the DB.

Mon, Oct 16, 3:29 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added a comment to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI.

Can you give some more details about this?
Where is this happening?
Where are you trying to connect to?
Which user are you using to connect to the DB?

Mon, Oct 16, 3:25 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 added projects to T178294: User log table creation on tools.labsdb failing intermittantly for IABot interactive UI: DBA, Tool-Database-Queries.

This appears to be a DB problem. This is making the tool and the bot notoriously unreliable. IABot workers are constantly crashing on different wikis due to DB failures.

Mon, Oct 16, 2:45 PM · Data-Services, DBA, InternetArchiveBot
Cyberpower678 moved T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki) from Bugs to New feature on the InternetArchiveBot (v1.6) board.
Mon, Oct 16, 1:45 PM · InternetArchiveBot (v1.6)
Cyberpower678 moved T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki) from Unsorted to Bugs on the InternetArchiveBot (v1.6) board.
Mon, Oct 16, 1:45 PM · InternetArchiveBot (v1.6)
Cyberpower678 renamed T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki) from Deploy InternetArchiveBot on the Bengali Wikipedia (barwiki) to Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki).
Mon, Oct 16, 1:43 PM · InternetArchiveBot (v1.6)
Cyberpower678 edited projects for T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki), added: InternetArchiveBot (v1.6); removed InternetArchiveBot (v2.0).
Mon, Oct 16, 9:16 AM · InternetArchiveBot (v1.6)
Cyberpower678 triaged T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki) as High priority.
Mon, Oct 16, 9:15 AM · InternetArchiveBot (v1.6)
Restricted Application assigned T178273: Deploy InternetArchiveBot on the Bavarian Wikipedia (barwiki) to Cyberpower678.
Mon, Oct 16, 9:14 AM · InternetArchiveBot (v1.6)

Sun, Oct 15

MusikAnimal awarded T136147: Deploy InternetArchiveBot on the German Wikipedia (dewiki) a Like token.
Sun, Oct 15, 10:24 PM · InternetArchiveBot (v1.5), Internet-Archive
Cyberpower678 removed a project from T136147: Deploy InternetArchiveBot on the German Wikipedia (dewiki): Community-consensus-needed.

Community consensus acquired.

Sun, Oct 15, 9:41 PM · InternetArchiveBot (v1.5), Internet-Archive

Fri, Oct 13

Cyberpower678 created T178134: Request static ip for cyberbot Cloud VPS project.
Fri, Oct 13, 3:00 AM · Cloud-VPS (Quota-requests)
Cyberpower678 created T178133: Create a method for forwarding non-HTTP protocols to VMs without public IPs.
Fri, Oct 13, 2:56 AM · Cloud-VPS

Thu, Oct 12

Cyberpower678 added a comment to T151446: Deploy InternetArchiveBot on the Ukrainian Wikipedia (ukwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:24 PM · InternetArchiveBot (v2.0), Internet-Archive, WMUA-Hackathons
Cyberpower678 added a comment to T169464: Deploy InternetArchiveBot on the Waray-Waray Wikipedia (warwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:23 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169465: Deploy InternetArchiveBot on the Polish Wikipedia (plwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:23 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169466: Deploy InternetArchiveBot on the Vietnamese Wikipedia (viwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:23 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169467: Deploy InternetArchiveBot on the Portuguese Wikipedia (ptwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:23 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169468: Deploy InternetArchiveBot on the Persian Wikipedia (fawiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:23 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169468: Deploy InternetArchiveBot on the Persian Wikipedia (fawiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:22 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169469: Deploy InternetArchiveBot on the Catalan Wikipedia (cawiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:22 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169470: Deploy InternetArchiveBot on the Arabic Wikipedia (arwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:22 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T169477: Deploy InternetArchiveBot on the Urdu Wikipedia (urwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:22 PM · InternetArchiveBot (v2.0), Internet-Archive
Cyberpower678 added a comment to T169482: Deploy InternetArchiveBot on the Bengali Wikipedia (bnwiki).

Deployment has been delayed in favor of development of InternetArchiveBot v2.0 which will be using a new global engine to support wikis more easily.

Thu, Oct 12, 1:21 PM · InternetArchiveBot (v2.0), Bengali-Sites
Cyberpower678 moved T151446: Deploy InternetArchiveBot on the Ukrainian Wikipedia (ukwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0), Internet-Archive, WMUA-Hackathons
Cyberpower678 moved T169464: Deploy InternetArchiveBot on the Waray-Waray Wikipedia (warwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169465: Deploy InternetArchiveBot on the Polish Wikipedia (plwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169466: Deploy InternetArchiveBot on the Vietnamese Wikipedia (viwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169467: Deploy InternetArchiveBot on the Portuguese Wikipedia (ptwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169468: Deploy InternetArchiveBot on the Persian Wikipedia (fawiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169469: Deploy InternetArchiveBot on the Catalan Wikipedia (cawiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169470: Deploy InternetArchiveBot on the Arabic Wikipedia (arwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169477: Deploy InternetArchiveBot on the Urdu Wikipedia (urwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0), Internet-Archive
Cyberpower678 moved T169482: Deploy InternetArchiveBot on the Bengali Wikipedia (bnwiki) from Unsorted to New feature on the InternetArchiveBot (v2.0) board.
Thu, Oct 12, 1:16 PM · InternetArchiveBot (v2.0), Bengali-Sites
Cyberpower678 moved T169477: Deploy InternetArchiveBot on the Urdu Wikipedia (urwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0), Internet-Archive
Cyberpower678 moved T169482: Deploy InternetArchiveBot on the Bengali Wikipedia (bnwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0), Bengali-Sites
Cyberpower678 moved T169470: Deploy InternetArchiveBot on the Arabic Wikipedia (arwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169469: Deploy InternetArchiveBot on the Catalan Wikipedia (cawiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T151446: Deploy InternetArchiveBot on the Ukrainian Wikipedia (ukwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0), Internet-Archive, WMUA-Hackathons
Cyberpower678 moved T169468: Deploy InternetArchiveBot on the Persian Wikipedia (fawiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169467: Deploy InternetArchiveBot on the Portuguese Wikipedia (ptwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169466: Deploy InternetArchiveBot on the Vietnamese Wikipedia (viwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169465: Deploy InternetArchiveBot on the Polish Wikipedia (plwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 moved T169464: Deploy InternetArchiveBot on the Waray-Waray Wikipedia (warwiki) from v1.6 to v2.0 on the InternetArchiveBot board.
Thu, Oct 12, 1:15 PM · InternetArchiveBot (v2.0)
Cyberpower678 added a comment to T177907: {{webarchive |format=...}}.

[

    "{{Article}}",
    "{{Bokref}}",
    "{{Cita DANFS}}",
    "{{Cita Enciclopedia Católica}}",
    "{{Cita TV}}",
    "{{Cita audio}}",
    "{{Cita conferencia}}",
    "{{Cita conferenza}}",
    "{{Cita enciclopedia}}",
    "{{Cita entrevista}}",
    "{{Cita episodio}}",
    "{{Cita grupo de noticias}}",
    "{{Cita historieta}}",
    "{{Cita immagine}}",
    "{{Cita juicio}}",
    "{{Cita libro}}",
    "{{Cita lista de correo}}",
    "{{Cita mapa}}",
    "{{Cita news}}",
    "{{Cita notas audiovisual}}",
    "{{Cita noticia}}",
    "{{Cita pubblicazione}}",
    "{{Cita publicación}}",
    "{{Cita tesis}}",
    "{{Cita testo}}",
    "{{Cita visual}}",
    "{{Cita vídeo}}",
    "{{Cita web}}",
    "{{Citar web}}",
    "{{Citation step free tube map}}",
    "{{Citation}}",
    "{{Cite AV media notes}}",
    "{{Cite AV media}}",
    "{{Cite Americana}}",
    "{{Cite Catholic Encylopedia}}",
    "{{Cite DVD notes}}",
    "{{Cite Hansard}}",
    "{{Cite PH act}}",
    "{{Cite act}}",
    "{{Cite book}}",
    "{{Cite comics image lrg}}",
    "{{Cite comics image}}",
    "{{Cite comic}}",
    "{{Cite constitution}}",
    "{{Cite conference}}",
    "{{Cite encyclopedia}}",
    "{{Cite episode}}",
    "{{Cite interview}}",
    "{{Cite journal}}",
    "{{Cite letter}}",
    "{{Cite magazine}}",
    "{{Cite mailing list}}",
    "{{Cite map}}",
    "{{Cite music release notes}}",
    "{{Cite newsgroup}}",
    "{{Cite news}}",
    "{{Cite podcast}}",
    "{{Cite press release}}",
    "{{Cite quick}}",
    "{{Cite report}}",
    "{{Cite serial}}",
    "{{Cite sign}}",
    "{{Cite speech}}",
    "{{Cite sports-reference}}",
    "{{Cite techreport}}",
    "{{Cite thesis}}",
    "{{Cite tweet}}",
    "{{Cite video}}",
    "{{Cite vob}}",
    "{{Cite web}}",
    "{{Citeer boek}}",
    "{{Citeer encyclopedie}}",
    "{{Citeer journal}}",
    "{{Citeer nieuws}}",
    "{{Citeer tijdschrift}}",
    "{{Citeer web}}",
    "{{Cytuj stronę}}",
    "{{Documentación cita}}",
    "{{Gazette WA}}",
    "{{Harvard reference}}",
    "{{Harvrefcol}}",
    "{{ITIS}}",
    "{{Internetquelle}}",
    "{{Kilde AV-medium}}",
    "{{Kilde artikkel}}",
    "{{Kilde avhandling}}",
    "{{Kilde avis}}",
    "{{Kilde bok}}",
    "{{Kilde konferanse}}",
    "{{Kilde oppslagsverk}}",
    "{{Kilde pressemelding}}",
    "{{Kilde www}}",
    "{{Lien conférence}}",
    "{{Lien vidéo}}",
    "{{Lien web}}",
    "{{Obra citada}}",
    "{{Pressmeddelanderef}}",
    "{{Tidningsref}}",
    "{{Tidskriftsref}}",
    "{{Vancite book}}",
    "{{Vancite journal}}",
    "{{Vancite news}}",
    "{{Vancite web}}",
    "{{Vcite book}}",
    "{{Vcite journal}}",
    "{{Vcite news}}",
    "{{Vcite web}}",
    "{{Web reference}}",
    "{{Webbref}}"
]
Thu, Oct 12, 12:50 AM · InternetArchiveBot (v1.5)
Cyberpower678 added a comment to T177907: {{webarchive |format=...}}.

Yes please

Thu, Oct 12, 12:49 AM · InternetArchiveBot (v1.5)

Wed, Oct 11

Cyberpower678 closed T177907: {{webarchive |format=...}} as Resolved.
Wed, Oct 11, 9:57 PM · InternetArchiveBot (v1.5)
Cyberpower678 added a comment to T177907: {{webarchive |format=...}}.

The template Gazette WA is not a recognized template and was ignored. Added in v1.5.7.

Wed, Oct 11, 9:51 PM · InternetArchiveBot (v1.5)
Cyberpower678 closed T177182: Spammy notices should go away as Declined.

There doesn't appear to be a consensus to remove the notices.

Wed, Oct 11, 9:02 PM · Community-consensus-needed, InternetArchiveBot
Cyberpower678 moved T177907: {{webarchive |format=...}} from Other tasks related to IABot to v1.5 on the InternetArchiveBot board.
Wed, Oct 11, 9:01 PM · InternetArchiveBot (v1.5)
Cyberpower678 set the image for InternetArchiveBot (v2.0) to F10148698: profile.
Wed, Oct 11, 9:00 PM
Cyberpower678 created InternetArchiveBot (v2.0).
Wed, Oct 11, 9:00 PM
Cyberpower678 added a comment to T176579: Investigate supporting MMS protocol on the checkIfDead class.

https://github.com/wikimedia/DeadlinkChecker/pull/25

Wed, Oct 11, 2:14 AM · InternetArchiveBot (v1.6)

Tue, Oct 10

Cyberpower678 added a comment to T176579: Investigate supporting MMS protocol on the checkIfDead class.

Unsupported protocols will return null instead of a boolean in v1.5.

Tue, Oct 10, 11:58 PM · InternetArchiveBot (v1.6)
Cyberpower678 added a comment to T176579: Investigate supporting MMS protocol on the checkIfDead class.

The checkIfDead class will support MMS and RTSP in v1.5

Tue, Oct 10, 4:36 PM · InternetArchiveBot (v1.6)
Cyberpower678 added a comment to T176562: Autogenerate false positive reports when the bot detects it's been reverted.

This is now refined in v1.6beta and is ready for testing.

Tue, Oct 10, 2:31 AM · InternetArchiveBot (v1.6)

Sat, Oct 7

Cyberpower678 closed T177185: {{cite constitution}} as Resolved.

Added cite constitution to the list if only to stop the bug.

Sat, Oct 7, 5:40 PM · InternetArchiveBot (v1.5)
Cyberpower678 closed T177002: IABotManagementConsole: Wrong encoding in summary line as Declined.

There is insufficient information provided to debug this problem.

Sat, Oct 7, 5:21 PM · InternetArchiveBot (v1.5)
Cyberpower678 added a comment to T177185: {{cite constitution}}.

Pinging @Green_Cardamom

Sat, Oct 7, 5:20 PM · InternetArchiveBot (v1.5)
Cyberpower678 closed T177615: Bracketed links with no space being misread as Resolved.

Fixed in v1.5.5

Sat, Oct 7, 5:20 PM · InternetArchiveBot (v1.5)

Fri, Oct 6

Cyberpower678 triaged T177457: Handle {{refbegin}} ... {{refend}} as Normal priority.
Fri, Oct 6, 9:11 PM · InternetArchiveBot (v1.6)
Cyberpower678 moved T177457: Handle {{refbegin}} ... {{refend}} from Unsorted to New feature on the InternetArchiveBot (v1.6) board.
Fri, Oct 6, 9:11 PM · InternetArchiveBot (v1.6)
Cyberpower678 added a comment to T177002: IABotManagementConsole: Wrong encoding in summary line.

@Iniquity pinging you for a response. I cannot proceed without your input.

Fri, Oct 6, 8:29 PM · InternetArchiveBot (v1.5)
Cyberpower678 added a comment to T177185: {{cite constitution}}.

What would you like me to do? Perhaps repurpose cite constitution to support the archive parameters?

Fri, Oct 6, 8:28 PM · InternetArchiveBot (v1.5)
Cyberpower678 triaged T177185: {{cite constitution}} as Normal priority.

It does, but they've been moved to a global file internally favoring the multi language engine. Cite constitution is not listed as a supported template, so the URL was simply seen as a plain URL.

Fri, Oct 6, 8:28 PM · InternetArchiveBot (v1.5)
Cyberpower678 triaged T177615: Bracketed links with no space being misread as Normal priority.
Fri, Oct 6, 8:26 PM · InternetArchiveBot (v1.5)
Cyberpower678 closed T176822: Internet Archive Bot has problems marking links as dead on ckb Wikipedia as Resolved.

Fixed in v1.5.5

Fri, Oct 6, 8:25 PM · InternetArchiveBot (v1.5)
Cyberpower678 added a comment to T176822: Internet Archive Bot has problems marking links as dead on ckb Wikipedia.

@Epine Firstly you broke the bot with your edit here: https://ckb.wikipedia.org/w/index.php?title=بەکارھێنەر:InternetArchiveBot/Dead-links.js&diff=543333&oldid=543318

Fri, Oct 6, 8:21 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T177185: {{cite constitution}} from Unsorted to Bugs on the InternetArchiveBot (v1.5) board.
Fri, Oct 6, 3:29 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T177615: Bracketed links with no space being misread from Unsorted to Bugs on the InternetArchiveBot (v1.5) board.
Fri, Oct 6, 3:29 PM · InternetArchiveBot (v1.5)
Cyberpower678 created T177615: Bracketed links with no space being misread.
Fri, Oct 6, 3:29 PM · InternetArchiveBot (v1.5)

Thu, Oct 5

Cyberpower678 moved T177185: {{cite constitution}} from Other tasks related to IABot to v1.5 on the InternetArchiveBot board.
Thu, Oct 5, 4:28 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T177457: Handle {{refbegin}} ... {{refend}} from Other tasks related to IABot to v1.6 on the InternetArchiveBot board.
Thu, Oct 5, 4:28 PM · InternetArchiveBot (v1.6)
Cyberpower678 moved T177513: Explore simple gadget creation to make using IABot even easier from Other tasks related to IABot to New feature on the InternetArchiveBot board.
Thu, Oct 5, 4:28 PM · InternetArchiveBot
Cyberpower678 updated subscribers of T177513: Explore simple gadget creation to make using IABot even easier.

I'm adding some typical involved participants as well as Legoktm who I previously partially consulted on this issue.

Thu, Oct 5, 4:26 PM · InternetArchiveBot
Restricted Application assigned T177513: Explore simple gadget creation to make using IABot even easier to Cyberpower678.
Thu, Oct 5, 4:25 PM · InternetArchiveBot
czar awarded T177457: Handle {{refbegin}} ... {{refend}} a 100 token.
Thu, Oct 5, 7:11 AM · InternetArchiveBot (v1.6)

Mon, Oct 2

Cyberpower678 added a comment to T176562: Autogenerate false positive reports when the bot detects it's been reverted.

A crude version now exists in v1.6beta but it's far from ready to be released to the public.

Mon, Oct 2, 1:10 AM · InternetArchiveBot (v1.6)

Sun, Oct 1

Cyberpower678 added a comment to T177182: Spammy notices should go away.

The bot is currently approved to run with these message. The last discussion regarding them had no consensus for change. If you feel the bot should stop leaving messages, please seek out a community discussion. This is a community bot.

Sun, Oct 1, 10:18 PM · Community-consensus-needed, InternetArchiveBot

Sat, Sep 30

Cyberpower678 triaged T176822: Internet Archive Bot has problems marking links as dead on ckb Wikipedia as Normal priority.
Sat, Sep 30, 6:39 PM · InternetArchiveBot (v1.5)
Cyberpower678 triaged T177002: IABotManagementConsole: Wrong encoding in summary line as Normal priority.
Sat, Sep 30, 6:39 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T176822: Internet Archive Bot has problems marking links as dead on ckb Wikipedia from Unsorted to Bugs on the InternetArchiveBot (v1.5) board.
Sat, Sep 30, 6:39 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T177002: IABotManagementConsole: Wrong encoding in summary line from Unsorted to Bugs on the InternetArchiveBot (v1.5) board.
Sat, Sep 30, 6:39 PM · InternetArchiveBot (v1.5)
Cyberpower678 moved T177002: IABotManagementConsole: Wrong encoding in summary line from Other tasks related to IABot to v1.5 on the InternetArchiveBot board.
Sat, Sep 30, 6:39 PM · InternetArchiveBot (v1.5)