Page MenuHomePhabricator
Create Task
Maniphest T237209

Alert SRE if Wikipedia Maps replication sync lag exceeds 1 day
Closed, InvalidPublic
Actions

Description

Wikipedia Maps sync lag has exceeded the expected time of 1 day. This is resulting in the updates on OSM such as name updates not being reflected. Screnshot of https://grafana.wikimedia.org/d/000000305/maps-performances?orgId=1&var-cluster=maps1 is enclosed.

I think the operations team should have an alert if the sync lags exceeds 1 day.

Grafana maps Screenshot from 2019-11-04 06-42-37.png (461×983 px, 42 KB)

Related Objects

Event Timeline

Arjunaraoc created this task.Nov 4 2019, 1:19 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 4 2019, 1:19 AM
Arjunaraoc renamed this task from Wikipedia Maps sync lag with OSM exceeded 5 days to Wikipedia Maps replication failed.Nov 4 2019, 1:24 AM
Arjunaraoc added a project: SRE.
Aklapper renamed this task from Wikipedia Maps replication failed to Alert SRE if Wikipedia Maps replication sync lag exceeds 1 day.Nov 4 2019, 8:31 AM
Aklapper removed a project: SRE.
Aklapper added a project: SRE.
• Mathew.onipe triaged this task as High priority.Nov 4 2019, 8:39 AM
• Mathew.onipe lowered the priority of this task from High to Medium.
• Mathew.onipe subscribed.

We have an alert for this. https://icinga.wikimedia.org/cgi-bin/icinga/extinfo.cgi?type=2&host=icinga1001&service=Maps+-+OSM+synchronization+lag+-+eqiad

• Mathew.onipe raised the priority of this task from Medium to High.Nov 4 2019, 8:41 AM
• Mathew.onipe added subscribers: Gehel, MSantos.
• Mathew.onipe removed subscribers: MSantos, Gehel.
• Mathew.onipe added a comment.Nov 4 2019, 8:44 AM

I'm closing this task as there are icinga alerts for osm sync

• Mathew.onipe closed this task as Invalid.Nov 4 2019, 8:44 AM
• Mathew.onipe added a comment.Nov 4 2019, 9:57 AM

see https://phabricator.wikimedia.org/T237228 for current OSM replication issues

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits