Page MenuHomePhabricator
Create Task
Maniphest T289077

Migrate Search team's prometheus-based alerts from Icinga to alert-manager
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

Incomplete list of [Search] alerts to migrate
  • WDQS lag + response times
  • blazegraph
  • mjolnir update failures

Details

SubjectRepoBranchLines +/-
wdqs/elastic: Remove icinga checks after moving to alertmanageroperations/puppetproduction+1 -94
search-platform: Port alerts from icingaoperations/alertsmaster+160 -17
Customize query in gerrit

Related Objects
Search...

Event Timeline

RKemper created this task.Aug 17 2021, 6:44 PM
RKemper removed a parent task: T288622: All Prometheus based alerts move from Icinga to alert manager exclusively.Aug 17 2021, 6:47 PM
RKemper added a parent task: T281454: Onboard teams with Prometheus-based alerts to AlertManager.
RKemper renamed this task from Migrate wdqs prometheus-based alerts from Icinga to alert-manager to Migrate Search team's prometheus-based alerts from Icinga to alert-manager.Aug 17 2021, 6:50 PM
RKemper updated the task description. (Show Details)
• MPhamWMF set the point value for this task to 5.Aug 23 2021, 3:43 PM
• MPhamWMF moved this task from Incoming to Ready for Dev -- SWE on the Discovery-Search (Current work) board.Aug 23 2021, 3:51 PM
lmata edited projects, added Observability-Alerting; removed SRE Observability (FY2021/2022-Q1).Oct 6 2021, 4:03 AM
lmata moved this task from Inbox to Radar on the Observability-Alerting board.
lmata subscribed.
• MPhamWMF subscribed.Dec 6 2021, 4:54 PM

Search will start working on this now, and will aim to have this done by end of Jan (unless we deliver a mini-holiday miracle)

EBernhardson claimed this task.Feb 15 2022, 5:36 PM
EBernhardson moved this task from Ready for Dev -- SWE to In Progress on the Discovery-Search (Current work) board.
gerritbot added a comment.Feb 15 2022, 6:45 PM

Change 762902 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/alerts@master] search-platform: Port alerts from icinga

https://gerrit.wikimedia.org/r/762902

gerritbot added a project: Patch-For-Review.Feb 15 2022, 6:45 PM
EBernhardson moved this task from In Progress to Needs review on the Discovery-Search (Current work) board.Feb 15 2022, 7:29 PM
gerritbot added a comment.Feb 28 2022, 7:16 PM

Change 766834 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/puppet@production] wdqs/elastic: Remove icinga checks after moving to alertmanager

https://gerrit.wikimedia.org/r/766834

gerritbot added a comment.Mar 3 2022, 6:06 PM

Change 762902 merged by jenkins-bot:

[operations/alerts@master] search-platform: Port alerts from icinga

https://gerrit.wikimedia.org/r/762902

gerritbot added a comment.Mar 3 2022, 6:31 PM

Change 766834 merged by Ryan Kemper:

[operations/puppet@production] wdqs/elastic: Remove icinga checks after moving to alertmanager

https://gerrit.wikimedia.org/r/766834

Maintenance_bot removed a project: Patch-For-Review.Mar 3 2022, 7:10 PM
RKemper moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Mar 3 2022, 7:34 PM
Gehel closed this task as Resolved.Mar 7 2022, 12:32 PM
fgiunchedi edited parent tasks, added: T288622: All Prometheus based alerts move from Icinga to alert manager exclusively; removed: T281454: Onboard teams with Prometheus-based alerts to AlertManager.Jul 11 2022, 12:08 PM
lmata moved this task from Radar to Done on the Observability-Alerting board.Jan 16 2023, 5:58 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits