Page MenuHomePhabricator

Implement etcd settings source
Closed, ResolvedPublic

Description

In order to load settings from etcd within SettingsBuilder, we'll need a new source implementation.

Requirements are roughly:

  1. Limit requests to etcd by.
    1. Implementing a naive cache key based on constructor arguments instead of relying on anything from the etcd response.
    2. Implementing a cache TTL of 10 seconds for this source only.
    3. Rely on the existing cache mitigation in CachedSource that uses probabilistic early expiration. (We will want to verify the effectiveness of this approach and possibly experiment with different values for EtcdSource::getExpiryWeight().
  2. Reuses or achieves parity with loading logic in EtcdConfig.

Event Timeline

dduvall changed the task status from Open to In Progress.Nov 30 2021, 7:02 PM
dduvall triaged this task as Medium priority.

Change 742786 had a related patch set uploaded (by Dduvall; author: Dduvall):

[mediawiki/core@master] Refactor cache TTL for SettingsBuilder sources

https://gerrit.wikimedia.org/r/742786

Change 742786 merged by jenkins-bot:

[mediawiki/core@master] Refactor cache TTL for SettingsBuilder sources

https://gerrit.wikimedia.org/r/742786

Change 744900 had a related patch set uploaded (by Dduvall; author: Dduvall):

[mediawiki/core@master] Make DnsSrvDiscoverer tolerant of discovery failure

https://gerrit.wikimedia.org/r/744900

Change 745125 had a related patch set uploaded (by Dduvall; author: Dduvall):

[mediawiki/core@master] WIP: Support settings loaded from etcd via SettingsBuilder

https://gerrit.wikimedia.org/r/745125

Change 755798 had a related patch set uploaded (by Dduvall; author: Dduvall):

[mediawiki/core@master] Support stale cached settings for failover

https://gerrit.wikimedia.org/r/755798

Change 756056 had a related patch set uploaded (by Dduvall; author: Dduvall):

[mediawiki/core@master] Use a mutex to prevent cold settings cache stampedes

https://gerrit.wikimedia.org/r/756056

Change 755798 merged by jenkins-bot:

[mediawiki/core@master] Support stale cached settings for failover

https://gerrit.wikimedia.org/r/755798

Change 756056 merged by jenkins-bot:

[mediawiki/core@master] Use a mutex to prevent settings cache stampedes

https://gerrit.wikimedia.org/r/756056

Change 759543 had a related patch set uploaded (by Dduvall; author: Dduvall):

[operations/mediawiki-config@master] beta: Discover etcd servers

https://gerrit.wikimedia.org/r/759543

Change 759543 merged by jenkins-bot:

[operations/mediawiki-config@master] beta: Discover etcd servers

https://gerrit.wikimedia.org/r/759543

Change 744900 merged by jenkins-bot:

[mediawiki/core@master] Simplify DnsSrvDiscoverer interface

https://gerrit.wikimedia.org/r/744900

Change 745125 merged by jenkins-bot:

[mediawiki/core@master] Support etcd as a source for SettingsLoader

https://gerrit.wikimedia.org/r/745125