See screenshot:
We would need to:
- stop immediately whatever task-creation workflow is producing them
- close/handle all hundred opened tasks
- rethink our usage of alerts-to-phabricator workflow
aborrero | |
Mar 28 2023, 9:15 AM |
F36930789: image.png | |
Mar 28 2023, 9:15 AM |
See screenshot:
We would need to:
via IRC @fgiunchedi mentioned that this might be related to some recent changes while migrating some alerts from icinga to alert-manager.
Change 903613 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):
[operations/alerts@master] sre: temp downgrade systemdunitfailed to warning, exclude wmcs
Change 903613 merged by Filippo Giunchedi:
[operations/alerts@master] sre: temp downgrade systemdunitfailed to warning, exclude wmcs
hey @Aklapper in the batch operation 3630 I made a mistake and set the ticket title instead of adding a comment. Not sure what to do with that, if try reverting somehow. On the other hand who cares, those tickets shouldn't have been created in the first place.
Change 904559 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):
[operations/puppet@production] alertmanager: disable phabricator task creation for WMCS alerts
Change 904559 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] alertmanager: update phabricator project for WMCS alerts
@aborrero: As you like. If you think it's really bad I could run a script to revert. But if nobody complains I'd just leave it as is.