@MarcoAurelio Is the bot being forced to /part the channel or /quit the network?

In T237109#5626847, @Masumrezarock100 wrote:

@MarcoAurelio Is the bot being forced to /part the channel or /quit the network?

It is disconnected by freenode from all channels, so I guess freenode just disconnects wikibugs from the network. See:

* wikibugs has quit (Excess Flood)
* wikibugs (~wikibugs2@wikimedia/bot/pywikibugs) has joined
* wikibugs has quit (Excess Flood)
* wikibugs (~wikibugs2@wikimedia/bot/pywikibugs) has joined
* ChanServ gives voice to wikibugs
* wikibugs has quit (Excess Flood)
* wikibugs (~wikibugs2@wikimedia/bot/pywikibugs) has joined
* ChanServ gives voice to wikibugs

That was from a channel wikibugs has remained silent the whole day; so it's safe to assume freenode doesn't just kick wikibugs out of the channel where it has "spoken too much, too quickly", but from the whole network.

A freenode staff can grant the bot exemption from auto kilne, I guess. One of the members of Wikimedia IRC group contact will need to contact a freenode staff for that.

@Masumrezarock100 is right, a Wikimedia IRC group contact will need to email iline@freenode.net (email might not be spelled correctly) to request an exemption. Even then (if I was told correctly) it's not guaranteed to be granted.

The bot is not being klined (we should indeed be exempt from that on a per-channel level), but being disconnected. Still problematic, but a little bit less so.

We have basic rate limiting set up (https://github.com/wikimedia/labs-tools-wikibugs2/blob/master/rqueue.py#L16 sets a max of one *event* every 2 seconds, which means one event from Gerrit and one from Phabricator. Each can then trigger a larger number of messages as events can get reported in multiple channels).

However, what might likely the issue here is hitting a secondary rate limit (i.e. not just the 'max N per second', but also a 'max M per Y time per IP address') as a result of a mass edit:

2019-11-01 14:22:17,453 - irc3.wikibugs - DEBUG - > USER wikibugs2 irc.freenode.net irc.freenode.net :Wikibugs v2.1, https://tools.wmflabs.org/wikibugs/ ,running on tools-sgeexec-0906.tools.eqiad.wmflabs
2019-11-01 14:22:55,685 - irc3.wikibugs - DEBUG - > USER wikibugs2 irc.freenode.net irc.freenode.net :Wikibugs v2.1, https://tools.wmflabs.org/wikibugs/ ,running on tools-sgeexec-0906.tools.eqiad.wmflabs
2019-11-01 14:23:07,486 - irc3.wikibugs - DEBUG - > USER wikibugs2 irc.freenode.net irc.freenode.net :Wikibugs v2.1, https://tools.wmflabs.org/wikibugs/ ,running on tools-sgeexec-0906.tools.eqiad.wmflabs
2019-11-01 14:23:53,450 - irc3.wikibugs - DEBUG - > USER wikibugs2 irc.freenode.net irc.freenode.net :Wikibugs v2.1, https://tools.wmflabs.org/wikibugs/ ,running on tools-sgeexec-0906.tools.eqiad.wmflabs

Then another disconnect around 18:00 UTC. So I think the rate limiting is largely working as intended; being disconnected in times of large numbers of messages is not a huge issue, and actually potentially beneficial (as messages remaining queued before and during the reconnect get dropped -- this prevents the bot from building up a potentially hours-long backlog).

test update

19:13 -!- wikibugs [~wikibugs2@wikimedia/bot/pywikibugs] has quit [Excess Flood]