Page MenuHomePhabricator

statsv: Counter and timer lines are discarded when a label value contains ".", "+" or "-"
Closed, ResolvedPublic

Description

WikimediaEvents' statsd.js sends DogStatsD label values made of A-Z, a-z, 0-9, "_", ".", "+" and "-", but statsv accepts ".", "+" and "-" only for histograms. It discards every counter and timer line with one of these characters in a label value, and logs this only at debug level, so the data is lost without any sign.

Technical notes

ACCEPTED_DOGSTATSD_COUNTER and ACCEPTED_DOGSTATSD_TIMING in statsv.py (lines 149-150) limit tags to [A-Za-z0-9_:,]. ACCEPTED_DOGSTATSD_HISTOGRAM (line 152, from T383563) already allows [A-Za-z0-9_:,.+-], so statsd_exporter already receives these characters in production without problems (for example le:0.3, le:-1, le:+Inf). The code comment cites the OpenMetrics limits for metric and label names, but those limits do not apply to label values.

Evidence from production (Thanos, 2026-09-24): of 2,799 statsv label values, none sent from a browser contains ".", "+" or "-"; the only matches are statsd_exporter build-info values such as non-git. VisualEditor Edit Check's mediawiki_editcheck_rules_total lines (community rule IDs such as LLM-user-comms, british-english) never arrive, while the lines around them in the same beacon do. The llm-paste check is absent from every editcheck metric, and hyphenated feedback reasons such as common-knowledge are lost too. Any other extension that sends such values is affected in the same way.

Patch: https://gerrit.wikimedia.org/r/q/I005a9e0512be61fbf12c32ea28663d9747550dfc widens both regexes to the histogram set and adds fixture cases, including values with "/" and a space that stay discarded.

Acceptance criteria

  • statsv forwards counter and timer lines whose label values contain ".", "+" or "-"
  • Lines with other characters in label values are still discarded
  • mediawiki_editcheck_rules_total series appear in Prometheus after the deploy

Event Timeline

Change #1344817 had a related patch set uploaded (by DLynch; author: DLynch):

[performance/statsv@master] statsv: Accept ".", "+" and "-" in counter and timer label values

https://gerrit.wikimedia.org/r/1344817

Change #1345172 had a related patch set uploaded (by Krinkle; author: Krinkle):

[performance/statsv@master] statsv: Reject dogstatsd_message longer than 5000 chars

https://gerrit.wikimedia.org/r/1345172

Change #1344817 merged by jenkins-bot:

[performance/statsv@master] statsv: Accept ".", "+" and "-" in counter and timer label values

https://gerrit.wikimedia.org/r/1344817

Change #1345172 merged by jenkins-bot:

[performance/statsv@master] statsv: Reject dogstatsd_message longer than 5000 chars

https://gerrit.wikimedia.org/r/1345172

Mentioned in SAL (#wikimedia-operations) [2026-09-25T18:27:18Z] <krinkle@deploy1003> Started deploy [statsv/statsv@df3ebff]: T439183: Accept dot, plus, hyphen in label values

Mentioned in SAL (#wikimedia-operations) [2026-09-25T18:27:29Z] <krinkle@deploy1003> Finished deploy [statsv/statsv@df3ebff]: T439183: Accept dot, plus, hyphen in label values (duration: 00m 11s)