Page MenuHomePhabricator

Turn up new eqiad-esams wave (Level3)
Closed, ResolvedPublic

Description

We've procured a new eqiad-esams wave. This task will track its implementation.

Event Timeline

faidon created this task.Jun 1 2016, 5:17 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJun 1 2016, 5:17 PM
faidon moved this task from Backlog to In Progress on the netops board.Jun 1 2016, 5:17 PM
faidon added a comment.Jun 1 2016, 5:46 PM

Spoke with the Level3 representative just now. Estimated delivery of the circuit is July 8th.

faidon moved this task from In Progress to Watching on the netops board.Jun 1 2016, 5:47 PM

The esams side is apparently ready, we're waiting for the LOA; the eqiad side is being actively worked on.

Commit date is the 26th, firm date is the 22nd, optimistic delivery is on the 18th.

faidon created subtask Unknown Object (Task).Jul 22 2016, 6:41 PM
faidon created subtask Unknown Object (Task).

We finally got the LOA. Subtasks for the cross-connect (both protected under S4) have been opened for Equinix/EvoSwitch respectively.

In the meantime, I allocated ports and IPv4/IPv6 subnets for the link and setup DNS with 0a1872b.

BFD & IGP is still pending and will follow only after we have confirmed that the link works (and we have the latency to put as the OSPF metric).

elukey added a subscriber: elukey.Jul 25 2016, 11:35 AM
mark closed subtask Unknown Object (Task) as Resolved.Jul 28 2016, 12:23 PM
faidon closed subtask Unknown Object (Task) as Resolved.Aug 14 2016, 5:08 PM
faidon closed this task as Resolved.Aug 14 2016, 5:15 PM

This took a while, with a lot of back and forths with Level3. They finally managed to patch their equipment in Ashburn on Aug 12th and in the Netherlands in Aug 13th (yesterday).

Latency is ~84ms, approximately 1.5ms more than our GTT link. I configured PIM, OSPF and OSPF3 (all with BFD) yesterday and ran a 10 minute test, after which I informed Level3 of the successful status of the link. The link has remained stable ever since and as such today I increased the metric on our other link to 1820 (820 because it's a ~82ms link + 1000 on virtue of not being a dedicated wave and thus downprefed).

I'll be closely monitoring it for the next few days. With the other link being still active and with both of them having BFD enabled, this is actually a pretty safe move (and is actually less risky than waiting to configure it).