Page MenuHomePhabricator

Decommission cookbook: lock per switch
Open, MediumPublic

Description

There is a risk of race condition during the switch config step when running the decom cookbook for multiple hosts at the same time.

To reduce that risk we should lock the cookbook per switch, if per switch is too complex, per row would be an ok middle-ground.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

To add a bit of contextual informations: this was triggered during T353449 and T353448 which were run seconds apart

If the delicate part is the call to configure_switch_interfaces() we can just change it's signature to require a lock instance and then wrap the delicate part with the acquired context manager so that we can have a per-switch lock that is independent on the cookbook that is calling it.

Yeah that would work too but might not be worth it as the cookbook main role is to run configure_switch_interfaces() and might be refactored in T344326: Junos module in Spicerack

ayounsi triaged this task as Medium priority.Feb 19 2024, 4:24 PM