Page MenuHomePhabricator

Serve volatile uri from local site
Closed, ResolvedPublic

Description

currently the volatile endpoint in puppet is served from the same location as the active puppetmaster, if the server serving the volatile end point goes down then the frontend puppetmaster in the other site dies as it is unable to keep up with all the timeouts to the backend endpoint. I believe all the data served from the volatile endpoint could be served from the local puppetmaster as it does not matter if nodes in one site receive data from volatile 15 minutes late (the volatile data is rsynced every 15 minutes).

Currently we have the following directories in volatile

  • GeoIP - maxmind geolocation data, a delay of 15 minutes is not harmful to this data
  • misc - maxmind geolocation data, a delay of 15 minutes is not harmful to this data
  • squid - outdated and can be removed
  • swift - swift rings, this data is site specific so it doesn't matter if sites are out of sync
  • tftpboot - this includes the boot images, as theses change very infrequently a delay of 15 minutes for propagation should be acceptable

Event Timeline

Change 542922 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] puppetmaster::frontend serve volatile uri from the locale site forntend

https://gerrit.wikimedia.org/r/542922

jbond renamed this task from Server volatile uri from local site. to Serve volatile uri from local site..Oct 14 2019, 12:43 PM
Reedy renamed this task from Serve volatile uri from local site. to Serve volatile uri from local site.Oct 14 2019, 2:18 PM
jijiki triaged this task as Medium priority.Oct 14 2019, 2:59 PM

Adding @BBlack, @ema, @Vgutierrez for explicit input/signoff wrt the GeoIP sub directory.

~15m delays should be ok for the GeoIP stuff, it was already sync'd to various consuming cache and DNS nodes over the ~30 minute splay window of puppet runs without issue.

Change 542922 merged by Jbond:
[operations/puppet@production] puppetmaster::frontend serve volatile uri from the locale site frontend

https://gerrit.wikimedia.org/r/542922

The volatile URI has been moved to both puppetmasters

jbond claimed this task.

This has now been implmented