Page MenuHomePhabricator

Spambots on #wikimedia-cloud
Closed, ResolvedPublic

Description

Lately, we have been having occasional spambots join #wikimedia-cloud and spam the channel with, so far, harmless messages.

Creating this task to track countermeasures.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 19 2018, 12:35 PM

Freenode Staff offered to have the Sigyn bot join the channel and keep an eye on spambots. This may not work against first offenders but should help to detect spambots and kill them from the whole network.

I've invited Sigyn to #wikimedia-cloud and provided Freenode with the following exemptions to avoid false positives: shinken-wm, icinga-wm, stashbot, wikibugs, wm-bot, jouncebot

Point of contact for Sigyn was niko but any staffer should be able to help in the future.

Same in many other channels such as #wikimedia-tech, #wikimedia-devrel etc.
#wikidata went for "Only registered (and authenticated) users will be heard when talking" today.

@Aklapper we discussed adding +r to the channel mode but the general consensus was that it would be too disruptive for new users at this moment (thus looking for alternatives unless the problem gets worse).

One thing you can do is +zq $~a to silence all unidentified users but allow channel ops to see their messages. You'll also want to +o Sigyn despite it having network oper status.

on #wikipedia-es we've set +q $~a so unregistered users cannot speak. We've also Sigyn in the channel with +o and BanBot helps issuing bans for the offending IPs for some time. So far it's working, although they keep comming :/. I also suggest adding wmopbot to the channels if you don't have it so wikimedia-ops can monitor in case nobody is around.

jrbs added a subscriber: jrbs.Sep 19 2018, 6:28 PM

Is Sigyn (kind of) solving the problem for now? The best-case is the bot enters, says one or two things, and is kicked. That's probably as good as you'll get without muting all unreg'd users as recommended above or just setting the channel as +r.

Unfortunately this is a Freenode-wide issue and all we can really do is damage control. Thankfully this is pretty harmless (a few months ago, the spam was much less harmless, so I guess we should count our blessings...).

Is Sigyn (kind of) solving the problem for now?

It's definitely made a lot of progress from what I can tell and is likely the best we can do. But there is still room for improvement in terms of FreeNode's handling of this. If they're going to auto-kill on certain phrases they should do so before such a message can be distributed to other users, not afterwards.

jrbs added a comment.Sep 19 2018, 6:31 PM

Is Sigyn (kind of) solving the problem for now?

It's definitely made a lot of progress from what I can tell and is likely the best we can do. But there is still room for improvement in terms of FreeNode's handling of this. If they're going to auto-kill on certain phrases they should do so before such a message can be distributed to other users, not afterwards.

Totally agree, though obviously not much we can do about that from our end. :/

bd808 added a subscriber: bd808.Sep 25 2018, 3:37 PM

Following the implementation of filters, spambots have adapted to just send pure garbage and now are replaying semi-intelligent messages that are sometimes hard to distinguish from legit messages. I'm thinking not even +q $~a will be enough because it takes way more time and conscious effort to read those messages now (if OPs were to help unregistered users in +q).

I thought about the following: set the channel to +r but also use +f to forward unregistered users to something like #wikimedia-cloud-unregistered where the topic could be set to instructions on how to register and/or a keyword to call for help (let's pray the spambots don't learn that). We can have a bot answering that too (unregistered user join #wikimedia-cloud, gets redirected, join #wikimedia-cloud-unregistered, bot sees join and sends instructions in private). This could work to deflect some of that traffic.

spambots have adapted to just send pure garbage and now are replaying semi-intelligent messages that are sometimes hard to distinguish from legit messages.

They are gonna pass the Turing test soon.

Jokes aside, I don't think there is anything preventing the spambots from registering a freenode account is there?

Registering a freenode account requires email verification, and freenode is very picky about what email providers they accept (no temp/burner email accounts or email services without anti-spam measures on signup). In any case, it's a lot more work to have a botnet go through that process before spamming, so the spambots simply do not register accounts.

If you set a +r and +f, I would recommend #wikimedia-overflow, as it already has a topic with those instructions and a large chanop presence who can handle the messages as needed.

MediaWiki-General has gone +zq $~a now :(

I've set #wikimedia-tech and #wikimedia-dev to +rf #wikimedia-overflow. It's not an ideal solution but it is a solution, hopefully we'll be able to increase productivity in those channels. #wikimedia-overflow has a large op presence that can either invite users or solve the issues.

daniel added a subscriber: daniel.Sep 27 2018, 5:04 PM

#wikimedia-de-tech has the same problem. it's WMDE's chat for the software development teams.

Az1568 added a subscriber: Az1568.Sep 27 2018, 5:26 PM

We have taken the step of using the mode settings +iI $j:#wikimedia-invite-exempt and +f #wikimedia-overflow. These settings make the channel invite-only (+i), but then exempt any user able to join the #wikimedia-invite-exempt channel which is being used as a shared access control mechanism from invites (+I $j:#wikimedia-invite-exempt). At last check, that shared ACL is allowing anyone connecting with an authenticated account, using a cloaked gateway, or having a nick matching certain allowed patterns.

The +f #wikimedia-overflow setting will forward any new connections that are blocked into the #wikimedia-overflow channel where there are instructions for how to register an account and potential help for people who are struggling to get connected.

The volume of spam messages is down since these measures have been applied, but it is unclear if there is a direct correlation or if other changes have happened in the attacker's behavior. I am tempted to remove these settings, but not over the weekend when we will have fewer people around to monitor and respond to the spam if it returns.

It looks like the situation is much better now. I understand we're never "done" with fighting spam but does it seem reasonable to close this task for now? Have we seen any collateral damage?

bd808 added a comment.Oct 18 2018, 5:10 PM

I do not have concrete evidence of collateral damage, but I also do not think we have concrete evidence of the +iI $j:#wikimedia-invite-exempt addition making a significant change in the rate of bot spam. I am wondering if we should try removing that flag to see if anything gets worse.

+1 for removing it and seeing what it looks like now. I was wondering if the spammers had taken a break but didn't want to jinx it. Hopefully it's not as bad as it was before.

Yeah the spammers already found the way around the wikimedia-invite-exempt trick anyway.

jrbs added a comment.Oct 18 2018, 6:13 PM

+1 for removing it and seeing what it looks like now. I was wondering if the spammers had taken a break but didn't want to jinx it. Hopefully it's not as bad as it was before.

Agree. Worst-case scenario, it’s reenabled.

bd808 closed this task as Resolved.Oct 18 2018, 9:49 PM
[19:05] Mode is +Cfijn #wikimedia-overflow 5:5
[19:07] bd808 sets mode -i
[19:07] Mode is +Cfjn #wikimedia-overflow 5:5