We were actually just talking about this in the SLOs group last week (adding @Vgutierrez and @CDanis).

In theory this is supposed to "Just Work": all our SLOs are defined as traffic fractions, successful requests divided by total requests. So when a host, or a site, is depooled, it gets no traffic at all, and doesn't increment either the numerator or the denominator. (And, if a host is supposed to be depooled but is still getting user traffic, we do want that traffic to count.) That's why we typically don't take pooled status into account explicitly.

There are a few reasons that doesn't actually work out in practice in this case:

• Purge traffic is the big one. We still send purges to depooled hosts, so they still affect the metrics -- and we do eventually care about those purges succeeding, but we don't care until the host is repooled. We think this was what drove the SLI movement here.
• (Health checks are the other obvious example, but in this case they're such a tiny fraction of traffic that they're negligible.)
• We'd also like to be able to use depooled hosts for load testing, without impacting the error budget.

One major takeaway is that we should split the purges out of the existing Varnish and ATS SLOs -- we still care about them, but the semantics is different and it makes sense to track them in a separate SLO.

But, especially because of the loadtest scenario, it probably does make sense to be able to mark a host as SLO-exempt, analogous to a downtime. It might also make sense to tie that to pooled status specifically (or, if we decide not to do that, maybe we incorporate it in the downtime automation -- either way we don't want to add a separate step that can be forgotten about).

@Vgutierrez had some more thoughts on this today, I'll let him chime in.

For instance, a recent decommissioning of codfw cp nodes (T419753) left the legacy ats-be service unavailable and caused the (depooled!) nodes to increment varnish_sli_bad as there were no servers left in the site according to them

This also made us realize some impact on still in production hosts that were relying on ats-be for backend caching (the old hosts in codfw were using chashing to pick an ats-be instance.

Ideally, I'd like the ability to schedule downtime or exclude servers from SLOs during maintenance operations independently of pooling/depooling actions. This way, we can verify everything looks good before repooling the servers.