We need conftool (rOSCT operations-software-conftool), or at least the python-conftool library, on deployment masters.
This is needed in order for us to implement pooling/depooling from scap as in {D600}.
Description
Description
Details
Details
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| role::deployment::mediawiki: include ::profile::conftool::client | operations/puppet | production | +2 -1 |
Revisions and Commits
Revisions and Commits
| Restricted Differential Revision |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Legoktm | T67289 Use semantic versioning scheme for WMF (all) releases | |||
| Resolved | • GWicke | T102550 Use semantic versioning for services (for consistency with mediawiki core) | |||
| Resolved | • mmodell | T94620 [EPIC] The future of MediaWiki deployment: Tooling | |||
| Open | Feature | None | T22085 [scap] Local sync script on any individual server should be atomic | ||
| Resolved | None | T125629 Depool proxies temporarily while scap is ongoing to avoid taxing those nodes | |||
| Resolved | None | T104352 Make scap able to depool/repool servers via the conftool API | |||
| Resolved | Joe | T73212 Make it possible to quickly and programmatically pool and depool application servers | |||
| Resolved | None | T115899 Move scap target configuration to etcd | |||
| Resolved | Joe | T163565 Install conftool on deployment masters |
Event Timeline
Comment Actions
Change 349498 had a related patch set uploaded (by 20after4):
[operations/puppet@production] Include ::profile::conftool::client on deployment servers
Comment Actions
I don't think scap should interact with conftool by itself, unless it reproduces what the restart-<service> scripts are doing right now, which is:
- depool in conftool
- verify that all LVS servers have actually depooled the host, in case they didn't, wait until they're able to depool the server
- restart
- do the same thing when repooling
basically, this is to ensure that we don't hit the depool threshold of pybal by doing e.g. a rolling restart that runs too fast.
Comment Actions
@Joe: That all seems reasonable. I don't particularly want to duplicate logic in scap unless it makes the most sense for that logic to live in scap.
This task is mostly concerned with implementing deplooling for mediawiki deployments but the functionality in scap could handle the process with other services as well. If this isn't the right approach then I think we could use some guidance from you on how to get this working.
Comment Actions
So, our original usecase for conftool was T125629: Depool proxies temporarily while scap is ongoing to avoid taxing those nodes -- we shouldn't have to worry much about depooling the whole cluster.
But then T162209: Figure out how node limitation interacts with proxies came along, which makes me wonder if we actually want proxies to be normal apaches, or if they'd be better as a kind of "deployment server lite" that serves as proxies for **all* targets, not just MW (and so we don't have to pick proxies for each service, all would immediately benefit). If this is the case, T125629 is moot.
Comment Actions
Oh, duh other total obvious usecase I forgot: being able to pull our target list from etcd directly, instead of having to write the dsh files as an intermediate step.
Comment Actions
Fair enough, this is a legitimate problem (depooling scap proxies).
I added some comments on the patch, and merging the change you have here since it makes sense to have conftool configured on deployment servers.
Comment Actions
Change 349498 merged by Giuseppe Lavagetto:
[operations/puppet@production] role::deployment::mediawiki: include ::profile::conftool::client