Page MenuHomePhabricator

10% of flame graph samples attributed to unknown_unknown
Closed, ResolvedPublic

Description

The MediaWiki Entrypoint Profiling: One component dashboard in Grafana plots the Arc Lamp flame graph samples as collected in wmf-config and attributed to ProfilerContext::getHandlerMetricPrefix, which we set as:

mediawiki-core/ActionEntryPoint.php
ProfilingContext::singleton()->init( MW_ENTRY_POINT, $actionName );

Screenshot 2026-08-19 at 16.35.16.png (890×542 px, 58 KB)

This means 10% of the time spent on mw-web and mw-api servers is unattributed. That's quite a big telemetry gap.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

The data attributed to "unknown_unknown" is collected with the same internal breakdown and granularity as everything else, so we can drill into like like any other entyrpoint to see what traffic might attributed to it.

Screenshot 2026-08-19 at 16.38.48.png (2,758×504 px, 153 KB)

They're almost all POST requests, and spend their time in core_JobQueue. Browsing through entrypoints in operations/mediawiki-config (/w and /rpc) they all set MW_ENTRY_POINT, except one: rpc/RunSingleJob.php.

Change #1327219 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/mediawiki-config@master] RunSingleJob: Define MW_ENTRY_POINT for flamegraph sample attribution

https://gerrit.wikimedia.org/r/1327219

Change #1327219 merged by jenkins-bot:

[operations/mediawiki-config@master] RunSingleJob: Define MW_ENTRY_POINT for flamegraph sample attribution

https://gerrit.wikimedia.org/r/1327219

Mentioned in SAL (#wikimedia-operations) [2026-08-20T21:27:20Z] <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1327219|RunSingleJob: Define MW_ENTRY_POINT for flamegraph sample attribution (T435422)]]

Mentioned in SAL (#wikimedia-operations) [2026-08-20T21:29:26Z] <krinkle@deploy1003> krinkle: Backport for [[gerrit:1327219|RunSingleJob: Define MW_ENTRY_POINT for flamegraph sample attribution (T435422)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-08-20T21:35:50Z] <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1327219|RunSingleJob: Define MW_ENTRY_POINT for flamegraph sample attribution (T435422)]] (duration: 08m 30s)

Change #1327652 had a related patch set uploaded (by Krinkle; author: Krinkle):

[operations/mediawiki-config@master] RunSingleJob: Add ProfilingContext::init()

https://gerrit.wikimedia.org/r/1327652

Change #1327652 merged by jenkins-bot:

[operations/mediawiki-config@master] RunSingleJob: Add ProfilingContext::init()

https://gerrit.wikimedia.org/r/1327652

Mentioned in SAL (#wikimedia-operations) [2026-08-20T23:30:59Z] <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1327652|RunSingleJob: Add ProfilingContext::init() (T435422)]]

Mentioned in SAL (#wikimedia-operations) [2026-08-20T23:33:05Z] <krinkle@deploy1003> krinkle: Backport for [[gerrit:1327652|RunSingleJob: Add ProfilingContext::init() (T435422)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-08-20T23:43:04Z] <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1327652|RunSingleJob: Add ProfilingContext::init() (T435422)]] (duration: 12m 06s)

Krinkle claimed this task.
Krinkle triaged this task as Medium priority.

Out: unknown_unknown. In: wmf_RunSingleJob, wmf_static.

Screenshot 2026-08-20 at 17.17.57.png (2,186×622 px, 92 KB)