This will be a lot of trouble, but it might be worthwhile to write some automated tests that log in on a few tools hosts, verifying that access works where it should and /does not work where it should not/.
Equivalent tests could be written to double-check sudo policies.
https://wikitech.wikimedia.org/wiki/Incident_documentation/20160212-LabsSudoVulnerability
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | dcaro | T142421 Write some full-stack tests | |||
| Open | None | T127716 [cloudvps,toolforge.infra] Write some labs tests that monitor login and sudo permissions |
Removing the Sustainability (Incident Followup) tag.
If this is still a solution to a possible incident root cause please add the tag back and consider raising the task priority.