Page MenuHomePhabricator

Frequent dispatch problems from February 22 on
Closed, ResolvedPublic

Description

From February 22 on we had a huge increase in change dispatching related alerts until February 28 and the same happened from March 13 until March 14.

This, especially the second peak (March 13 and 14), correlates well with Research Bot's edit rate:

MariaDB [wikidatawiki_p]> SELECT LEFT(rc_timestamp, 8) AS day, COUNT(*) AS editCount, SUM(rc_new_len - rc_old_len) AS addedBytes FROM recentchanges WHERE rc_user = 2752938 GROUP BY day ORDER BY day DESC;
+----------+-----------+------------+
| day      | editCount | addedBytes |
+----------+-----------+------------+
| 20180315 |     21499 |  503473055 |
| 20180314 |     80131 | 1967198677 |
| 20180313 |      8516 |  210935987 |
| 20180306 |      5564 |  119459849 |
| 20180305 |      5407 |  104758366 |
| 20180304 |      3040 |   46759038 |
| 20180303 |      3847 |   64375729 |
| 20180302 |     12270 |  262194751 |
| 20180301 |      4341 |   78344021 |
| 20180228 |     29571 |  655748901 |
| 20180227 |     31077 |  764508347 |
| 20180226 |     46013 | 1028998913 |
| 20180225 |     39239 |  899337980 |
| 20180224 |     53646 | 1286406337 |
| 20180223 |     44302 | 1053010363 |
| 20180222 |     23731 |  542486466 |
| 20180221 |     26015 |  576737416 |
| 20180220 |     22374 |  473697546 |
| 20180219 |     24108 |  468238047 |
| 20180218 |     61516 | 1404120034 |
| 20180217 |     41251 |  940858733 |
| 20180216 |     63926 | 1468175747 |
| 20180215 |     60279 | 1395074072 |
| 20180214 |     55100 | 1274020552 |
| 20180213 |     41713 |  961220753 |
+----------+-----------+------------+
25 rows in set (27.96 sec)

I suspect that especially the huge changes done by Research Bot are way larger than the average Wikidata edit:

MariaDB [wikidatawiki_p]> SELECT SUM(rc_new_len - rc_old_len)/COUNT(*) AS avgEditDelta FROM recentchanges WHERE rc_user = 2752938;
+--------------+
| avgEditDelta |
+--------------+
|   22944.5350 |
+--------------+
1 row in set (9.06 sec)
MariaDB [wikidatawiki_p]> SELECT SUM(rc_new_len - rc_old_len)/COUNT(*) AS avgEditDelta FROM recentchanges WHERE rc_namespace = 0;
+--------------+
| avgEditDelta |
+--------------+
|    1473.1311 |
+--------------+
1 row in set (1 min 7.84 sec)

Event Timeline

hoo claimed this task.
hoo moved this task from Tasks to Done on the Wikidata-Ministry-Of-Magic board.

I asked the operator of ResearchBot to slow down and they did: https://www.wikidata.org/w/index.php?title=User_talk:Daniel_Mietchen&oldid=649478954#Please_slow_down

This solved the problem for now, we had no alerts since.