What/Why:
There are custom spans that were implemented last year (2025) sometime to emit granular spans at specific function calls within a trace. This is set behind the config flag createCustomSpans; for both the Evaluator and Orchestrator.
We turned this feature off temporarily while investigating service slowness and memory exhaustion. Today as SRE is looking into further support on service-utils (and AW being the first team to guinea-pig it), we want to turn this back on and verify that these spans are coming through correctly.
How:
Locate all of the functions which calls the custom span creation logic, possibly remove from a couple of heavy-weight functions(?), and set createCustomSpans to true in our deploy config.
Current areas that trigger custom span creation
in Orchestrator:
- evaluate.js
- execute.js
- fetchObject.js
- implementation.js
- validation.js
in Evaluator:
- Executor.js
