Proposed collection-side architecture:

• Collection
  • Puppet profile::opentelemetry::collector deployed on instrumented bare metal nodes
  • opentelemetry-collector running as DaemonSet on wikikube

both using trace-id aware load-balancing send to

• Aggregation
  • opentelemetry-gateway: opentelemetry-collector running as a standard Deployment on aux-k8s (sampling happens here for tail sampling)

Forwards to

• Storage
  • jaeger-collector running on aux-k8s, in charge of writing to opensearch.
    • jaeger-collector should be configured with --collector.otlp.enabled=true, and then will accept OTLP format traces via HTTP on port 4318 (see https://www.jaegertracing.io/docs/1.46/deployment/#collector)

There are probably some things I have not thought of in there, but it's a starting point.

Possible things to explore:

• Using the elasticsearch exporter directly at the opentelemetry-gateway level (can Jaeger exploit these afterwards?)

NB: This is a brain dump, discussion welcome :)