Page MenuHomePhabricator
Create Task
Maniphest T345698

[Epic] define a strategy around alerting for Data Platform SRE and implement it
Closed, DuplicatePublic
Actions

Details

SubjectRepoBranchLines +/-
Update the contact info for the wikireplica serversoperations/puppetproduction+6 -7
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 DuplicateNoneT345698 [Epic] define a strategy around alerting for Data Platform SRE and implement it
 OpenNoneT343234 Explore the use of Airflow notifiers for more flexible DAG failure handling
 ResolvedAntoine_QuhenT343232 Configure Airflow to send metrics to Prometheus
 ResolvedAntoine_QuhenT349532 [Data Quality] Implement Simple Monitoring Dashboard for Airflow Jobs
 ResolvedAntoine_QuhenT351792 Unblock Dockerfile syntax to build images with Gitlab trusted runner
 OpenNoneT346438 [Epic] Review alerting strategy for Data Platform SRE
 OpenNoneT309012 Migrate zookeeper prometheus checks from Icinga to Alertmanager
 OpenNoneT343761 Confirm TLS certificate monitoring is in place for Search Platform-owned domains
 ResolvedBTullisT342578 Ensure Data Platform SREs have a contact group in puppet/alerting
 ResolvedBTullisT344202 Create VictorOps config for new Data Platform SRE team
 OpenNoneT336574 Review alerting around Wikidata Query Service update pipeline
 DeclinedNoneT346807 Review alerting around Search update pipeline
 OpenNoneT337055 Send a critical alert to data-engineering if produce_canary_events isn't running correctly
 OpenNoneT337052 Reduce IRC/alert noise associated with monitor_refine_ systemd timers from alertmanager
 OpenbkingT352783 Change data platform-related IRC channels to improve communication
 ResolvedbkingT357537 Alerts Review: determine if we can use Prometheus to alert based on historical datasets
 ResolvedBTullisT358205 Investigate late/delayed Airflow task failure notifications
 OpenNoneT349772 Create dashboards/alerts for new Cirrus Streaming Updater
 ResolvedbkingT359213 Adapt Flink-related rdf-streaming-updater alerts for Cirrus Streaming Updater
 ResolvedbkingT361862 Determine cause of unexpectedly high blackbox poller entries in wdqs nginx access logs
 OpenNoneT350499 Search Platform Airflow jobs: Identify dependencies and configure alerts
 ResolvedNoneT352921 Review alerts for miscweb-hosted domains commons-query.wikimedia.org and query.wikidata.org

Event Timeline

Gehel created this task.Sep 6 2023, 8:43 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 6 2023, 8:43 AM
Gehel added a project: Epic.Sep 6 2023, 8:43 AM
Gehel moved this task from Incoming to Epics on the Data-Platform-SRE board.
Gehel added a subtask: T343234: Explore the use of Airflow notifiers for more flexible DAG failure handling.Sep 6 2023, 8:45 AM
Gehel added a subtask: T343232: Configure Airflow to send metrics to Prometheus.
BTullis mentioned this in T346438: [Epic] Review alerting strategy for Data Platform SRE.Sep 20 2023, 8:29 AM
Gehel triaged this task as High priority.Oct 11 2023, 8:47 AM
Gehel moved this task from Epics to Quarterly Goals on the Data-Platform-SRE board.Oct 16 2023, 1:06 PM
Gehel moved this task from Quarterly Goals to Epics on the Data-Platform-SRE board.
bking added a subtask: T346438: [Epic] Review alerting strategy for Data Platform SRE.Nov 3 2023, 3:20 PM
Gehel added a subtask: T350499: Search Platform Airflow jobs: Identify dependencies and configure alerts.Nov 6 2023, 7:26 PM
BTullis subscribed.Nov 7 2023, 4:09 PM

Is this a duplicate of T346438: [Epic] Review alerting strategy for Data Platform SRE?

gerritbot added a comment.Nov 9 2023, 4:56 PM

Change 973203 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/puppet@production] Update the contact info for the wikireplica servers

https://gerrit.wikimedia.org/r/973203

gerritbot added a project: Patch-For-Review.Nov 9 2023, 4:56 PM
gerritbot added a comment.Nov 16 2023, 10:01 AM

Change 973203 merged by Btullis:

[operations/puppet@production] Update the contact info for the wikireplica servers

https://gerrit.wikimedia.org/r/973203

Maintenance_bot removed a project: Patch-For-Review.Nov 16 2023, 10:10 AM
bking closed subtask T352921: Review alerts for miscweb-hosted domains commons-query.wikimedia.org and query.wikidata.org as Resolved.Dec 18 2023, 7:36 PM
lbowmaker closed subtask T343232: Configure Airflow to send metrics to Prometheus as Resolved.Jan 30 2024, 12:19 PM
BTullis closed this task as a duplicate of T346438: [Epic] Review alerting strategy for Data Platform SRE.Feb 8 2024, 10:25 AM
bking added a subtask: T349772: Create dashboards/alerts for new Cirrus Streaming Updater.Mar 5 2024, 8:27 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL