Page MenuHomePhabricator

Toolforge: Replag on s5 is >22 hours
Closed, ResolvedPublic

Description

When I want to see details about my contributions on Serbian Wikipedia (sr.wikipedia.org) via "Edit Counter" option of XTools tool, I get warning about replication lag.

I've checked replication lag via replag.toolforge.org and I have found that replication for s5 wikis (s5.dblist) is >22 hours.

Related Objects

Event Timeline

Kizule created this task.Jul 19 2020, 6:42 PM
Restricted Application added a subscriber: Petar.petkovic. · View Herald TranscriptJul 19 2020, 6:42 PM
Kizule triaged this task as High priority.Jul 19 2020, 6:48 PM
Wiki13 added a subscriber: Wiki13.Jul 19 2020, 7:06 PM

See T258336. Database host crashed yesterday, which in turn creates replication lag. See below for snippit from that task

Mentioned in SAL (#wikimedia-operations) [2020-07-18T20:52:50Z] <marostegui> Due to db1082 crash there will be replication lag on s5 on labsdb hosts - T258336

JJMC89 raised the priority of this task from High to Needs Triage.Jul 19 2020, 7:06 PM
JJMC89 edited projects, added Data-Services; removed Toolforge, XTools.
bd808 moved this task from Backlog to Wiki replicas on the Data-Services board.Jul 19 2020, 11:33 PM

Now this happens: In few seconds I can see that replication lag is 0 and second after, replication lag is >11 hours. And it happens constantly now.

Replication lag on analytics hosts is now 0.
On the web service it is still 7h behind, but catching up nicely

Thanks @Marostegui. I will close this task when replication completly recovers.

Kizule closed this task as Resolved.Jul 20 2020, 6:14 AM
Kizule assigned this task to Marostegui.

Closing now.