Following the discovery of a powerdns configuration issue during T331541: 14 March 2023 eqiad Service repooling, we discovered that cookbooks that deal with discovery pooling and depooling only check authdns servers.
Ideally, they should check recdns from every site that the correct IPs are returned after wiping cache.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Clement_Goubert | T327920 March 2023 Datacenter Switchover | |||
| Open | Clement_Goubert | T328907 Post March 2023 Datacenter Switchover Tasks | |||
| Resolved | Clement_Goubert | T331541 14 March 2023 eqiad Service repooling | |||
| Open | None | T332009 Cookbooks that do DNS discovery change should check recdns |
Event Timeline
Comment Actions
I am not sure what would be the goal of checking the dns recursors in that situation, as far as running the cookbook would be concerned. Say we wipe the cache and the recdns isn't in our desired state; what can the cookbook do as a corrective measure? Does it make sense to throw an error?
I think it could be more interesting to have a function to go check the state of the dns recursors compared to etcd to run as a monitoring check maybe.