Page MenuHomePhabricator

ext:StopForumSpam config seemingly broke channel:StopForumSpam within beta-logs.wmcloud.org
Closed, ResolvedPublic

Description

After setting ext:StopForumSpam to "enforce" on the beta cluster (T304111) [basically removing $wgSFSReportOnly so that it defaults to false per extension.json], channel:StopForumSpam seemed to stop working within beta-logs.wmcloud.org, pretty much at the same time:

beta-logs.png (293×671 px, 41 KB)

We're still getting logs on the beta cluster (/srv/mw-log) for this channel, and they seem fine. In fact, the extension seemed to work fine in "enforce" mode according to the logs for that timeframe. It just broke them making it over to beta-logs for some reason, and then remained broken after I reverted the config change, since I didn't know what was going on. The only new logging change which should have been introduced is this block, since it comes after the reporty-only early return. But that seems pretty innocuous on its face to me, so I'm not sure how that would've caused this issue.

Event Timeline

Change 773285 had a related patch set uploaded (by Cwhite; author: Cwhite):

[operations/puppet@production] profile: Rsyslog omkafka configs use new ca bundle

https://gerrit.wikimedia.org/r/773285

colewhite changed the task status from Open to In Progress.Mar 23 2022, 4:52 PM
colewhite claimed this task.
colewhite triaged this task as High priority.
colewhite added subscribers: elukey, colewhite.

Change appears related to the shift to PKI in beta. FYI @elukey

Change 773285 merged by Cwhite:

[operations/puppet@production] profile: Rsyslog omkafka configs use new ca bundle

https://gerrit.wikimedia.org/r/773285

The logs are flowing again after the rsyslog reconfiguration.