Page MenuHomePhabricator
Create Task
Maniphest T395228

Test katran forwarding plane on lvs1013
Closed, ResolvedPublic
Actions

Description

We currently keep lvs1013 as a liberica canary performing the load balancing of ncredir@eqiad. This is the perfect candidate for the initial katran tests

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Vgutierrez created this task.May 26 2025, 8:31 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 26 2025, 8:31 AM
Vgutierrez triaged this task as Medium priority.May 26 2025, 8:31 AM
gerritbot added a comment.May 26 2025, 8:34 AM

Change #1150587 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] hiera: Depool lvs1013 before switching to katran

https://gerrit.wikimedia.org/r/1150587

gerritbot added a project: Patch-For-Review.May 26 2025, 8:34 AM
gerritbot added a comment.May 26 2025, 10:58 AM

Change #1150626 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] hiera: Use katran in lvs1013

https://gerrit.wikimedia.org/r/1150626

gerritbot added a comment.May 28 2025, 10:37 AM

Change #1151644 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] liberica: Don't set forwarding_cores/numa_node for katran

https://gerrit.wikimedia.org/r/1151644

gerritbot added a comment.Jun 3 2025, 10:56 AM

Change #1151644 merged by Vgutierrez:

[operations/puppet@production] liberica: Don't set forwarding_cores/numa_node for katran

https://gerrit.wikimedia.org/r/1151644

Stashbot added a comment.Jun 4 2025, 10:00 AM

Mentioned in SAL (#wikimedia-operations) [2025-06-04T10:00:07Z] <vgutierrez> depool lvs1013 before switching to katran - T395228

gerritbot added a comment.Jun 4 2025, 10:00 AM

Change #1150587 merged by Vgutierrez:

[operations/puppet@production] hiera: Depool lvs1013 before switching to katran

https://gerrit.wikimedia.org/r/1150587

Stashbot added a comment.Jun 4 2025, 10:04 AM

Mentioned in SAL (#wikimedia-operations) [2025-06-04T10:04:09Z] <vgutierrez> upload liberica 0.15 to bookwork-wikimedia (apt.wm.o) - T395228

ops-monitoring-bot added a comment.Jun 4 2025, 10:05 AM

Icinga downtime and Alertmanager silence (ID=d0bc6853-e370-4194-a010-37ddc420e227) set by vgutierrez@cumin1002 for 1 day, 0:00:00 on 1 host(s) and their services with reason: switching to katran

lvs1013.eqiad.wmnet
gerritbot added a comment.Jun 4 2025, 10:14 AM

Change #1150626 merged by Vgutierrez:

[operations/puppet@production] hiera: Use katran in lvs1013

https://gerrit.wikimedia.org/r/1150626

Maintenance_bot removed a project: Patch-For-Review.Jun 4 2025, 10:31 AM
gerritbot added a comment.Jun 5 2025, 1:25 PM

Change #1154027 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^2 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154027

gerritbot added a project: Patch-For-Review.Jun 5 2025, 1:25 PM
Stashbot added a comment.Jun 5 2025, 1:48 PM

Mentioned in SAL (#wikimedia-operations) [2025-06-05T13:48:33Z] <vgutierrez> upload liberica 0.16 to bookworm-wikimedia (apt.wm.o) - T395228

gerritbot added a comment.Jun 5 2025, 1:52 PM

Change #1154033 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^2 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154033

gerritbot added a comment.Jun 5 2025, 2:14 PM

Change #1154027 merged by Vgutierrez:

[operations/puppet@production] Revert^2 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154027

ops-monitoring-bot added a comment.Jun 5 2025, 2:19 PM

Icinga downtime and Alertmanager silence (ID=8a04aff9-1dff-495f-85e9-18f315f11e24) set by vgutierrez@cumin1002 for 1 day, 0:00:00 on 1 host(s) and their services with reason: switching to katran

lvs1013.eqiad.wmnet
gerritbot added a comment.Jun 5 2025, 2:26 PM

Change #1154033 merged by Vgutierrez:

[operations/puppet@production] Revert^2 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154033

Maintenance_bot removed a project: Patch-For-Review.Jun 5 2025, 2:31 PM
gerritbot added a comment.Jun 6 2025, 11:07 AM

Change #1154268 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] liberica: Don't bind VIPs to lo interface with katran

https://gerrit.wikimedia.org/r/1154268

gerritbot added a project: Patch-For-Review.Jun 6 2025, 11:07 AM
gerritbot added a comment.Jun 6 2025, 12:46 PM

Change #1154268 merged by Vgutierrez:

[operations/puppet@production] liberica: Don't bind VIPs to lo interface with katran

https://gerrit.wikimedia.org/r/1154268

gerritbot added a comment.Jun 6 2025, 1:15 PM

Change #1154286 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^4 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154286

gerritbot added a comment.Jun 6 2025, 1:31 PM

Change #1154286 merged by Vgutierrez:

[operations/puppet@production] Revert^4 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154286

ops-monitoring-bot added a comment.Jun 6 2025, 1:40 PM

Icinga downtime and Alertmanager silence (ID=7a07b406-abbe-4b89-a057-69956a37122f) set by vgutierrez@cumin1003 for 1 day, 0:00:00 on 1 host(s) and their services with reason: switching to katran

lvs1013.eqiad.wmnet
gerritbot added a comment.Jun 6 2025, 1:42 PM

Change #1154294 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^4 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154294

gerritbot added a comment.Jun 6 2025, 2:25 PM

Change #1154294 merged by Vgutierrez:

[operations/puppet@production] Revert^4 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154294

Maintenance_bot removed a project: Patch-For-Review.Jun 6 2025, 2:31 PM
gerritbot added a comment.Jun 6 2025, 3:03 PM

Change #1154307 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^5 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154307

gerritbot added a project: Patch-For-Review.Jun 6 2025, 3:03 PM

Change #1154307 merged by Vgutierrez:

[operations/puppet@production] Revert^5 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154307

Maintenance_bot removed a project: Patch-For-Review.Jun 6 2025, 3:32 PM
gerritbot added a comment.Jun 9 2025, 8:47 AM

Change #1154768 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^6 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154768

gerritbot added a project: Patch-For-Review.Jun 9 2025, 8:47 AM
Stashbot added a comment.Jun 9 2025, 9:20 AM

Mentioned in SAL (#wikimedia-operations) [2025-06-09T09:20:36Z] <vgutierrez> upload liberica 0.17 to apt.wm.o (bookworm-wikimedia) - T395228

gerritbot added a comment.Jun 9 2025, 9:30 AM

Change #1154768 merged by Vgutierrez:

[operations/puppet@production] Revert^6 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154768

Stashbot added a comment.Jun 9 2025, 9:31 AM

Mentioned in SAL (#wikimedia-operations) [2025-06-09T09:31:17Z] <vgutierrez> depooling lvs1013 before switching ncredir@eqiad to katran based load balancing - T395228

Maintenance_bot removed a project: Patch-For-Review.Jun 9 2025, 9:31 AM
gerritbot added a comment.Jun 9 2025, 9:38 AM

Change #1154774 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^6 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154774

gerritbot added a project: Patch-For-Review.Jun 9 2025, 9:38 AM
gerritbot added a comment.Jun 9 2025, 9:48 AM

Change #1154774 merged by Vgutierrez:

[operations/puppet@production] Revert^6 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154774

Stashbot added a comment.Jun 9 2025, 10:16 AM

Mentioned in SAL (#wikimedia-operations) [2025-06-09T10:16:48Z] <vgutierrez> repooling lvs1013 handling ncredir@eqiad using katran based load balancing - T395228

gerritbot added a comment.Jun 9 2025, 10:17 AM

Change #1154779 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^7 "hiera: Use katran in lvs1013"

https://gerrit.wikimedia.org/r/1154779

gerritbot added a comment.Jun 9 2025, 10:19 AM

Change #1154779 abandoned by Vgutierrez:

[operations/puppet@production] Revert^7 "hiera: Use katran in lvs1013"

Reason:

wrong CR

https://gerrit.wikimedia.org/r/1154779

gerritbot added a comment.Jun 9 2025, 10:20 AM

Change #1154781 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] Revert^7 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154781

gerritbot added a comment.Jun 9 2025, 10:27 AM

Change #1154781 merged by Vgutierrez:

[operations/puppet@production] Revert^7 "hiera: Depool lvs1013 before switching to katran"

https://gerrit.wikimedia.org/r/1154781

Maintenance_bot removed a project: Patch-For-Review.Jun 9 2025, 10:31 AM
gerritbot added a comment.Jun 9 2025, 1:19 PM

Change #1154816 had a related patch set uploaded (by Vgutierrez; author: Vgutierrez):

[operations/puppet@production] liberica: Install xdp-tools on liberica LBs

https://gerrit.wikimedia.org/r/1154816

gerritbot added a project: Patch-For-Review.Jun 9 2025, 1:19 PM
gerritbot added a comment.Jun 9 2025, 2:02 PM

Change #1154816 merged by Vgutierrez:

[operations/puppet@production] liberica: Install xdp-tools on liberica LBs

https://gerrit.wikimedia.org/r/1154816

Maintenance_bot removed a project: Patch-For-Review.Jun 9 2025, 2:31 PM
herron subscribed.Jun 9 2025, 4:52 PM
Vgutierrez closed this task as Resolved.Jun 11 2025, 8:02 AM

lvs1013 has been balancing ncredir@eqiad using katran since 2025-06-09 at 10:30

ops-monitoring-bot added a comment.Jun 12 2025, 1:01 PM

Icinga downtime and Alertmanager silence (ID=5df8241b-6dee-4131-8bad-8d9a73984d08) set by vgutierrez@cumin1003 for 1 day, 0:00:00 on 1 host(s) and their services with reason: switching to katran

lvs7002.magru.wmnet
ops-monitoring-bot added a comment.Jun 16 2025, 8:31 AM

Icinga downtime and Alertmanager silence (ID=59c5d762-0532-415b-82b0-0ec6b72d478a) set by vgutierrez@cumin1002 for 1 day, 0:00:00 on 1 host(s) and their services with reason: switching to katran

lvs7001.magru.wmnet
Vgutierrez closed subtask T396227: unable to fetch forwarding plane metrics when using katran as Resolved.Jun 20 2025, 3:29 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits