In T320565 and T320551 we packaged the OpenTelemetry Collector as a .deb and also prepared some puppetization, but didn't actually deploy it to any roles or profiles or hosts.
So let's
- modify mediawiki::canary_appserver (the role used for mwdebug hosts) to include profile::opentelemetry::collector (possibly optionally based on hiera, if we want to not to deploy to all the canaries immediately, but I don't see too much reason for that)
- modify profile::tlsproxy::envoy to optionally report otel metadata to the local otel collector, configuring it with a tracing provider, similar to but different from the work done in T320563 on enabling it on the wikikube mesh (see also this patch)
- modify profile::tlsproxy::envoy to have a configurable sampling fraction to initiate traces
- set that sampling fraction to 1.0 on mwdebug*
Looking at the Envoy tracing configuration stanzas it seems like we would want to set random_sampling: 1.0 on debug hosts, and leave other values at their defaults.