Page MenuHomePhabricator

Fortify new API metrics method
Closed, ResolvedPublic

Description

Followup to T402074: Set up new prometheus label structure for the API monitoring there's a need to fortify and fix the unified stats method.

The original method intended to provide base setup with the ability to override the values, but the way it was written can also unintentionally allow for unregistered/unexpected new labels, which should not happen.

There is also an issue where some edge cases seem to not fill certain label values correctly. The new metrics should record the "hit" on the API module, so we don't want to *ignore* that error, but we cannot let it pass as-is because it creates warnings. We should make sure we log these with some clearly noted "error" state in the value so we can make sure we track things (in grafana) and see how many (and why) these types of API errors happen.

  • Make sure labels are set, and only values can be overridden.
  • Record empty label values as clearly noted "error" for tracking instead of skipping them.

Also see T411585: PHP Warning: Stats: (action_api_modules_hit_total): Stats: (action_api_modules_hit_total) Cannot associate label keys with label values - Not all initialized labels have an assigned value.

Event Timeline

Change #1215203 had a related patch set uploaded (by Mooeypoo; author: Mooeypoo):

[mediawiki/core@master] Followup Ie40b9e59a4: Fortify unified metrics method

https://gerrit.wikimedia.org/r/1215203

Change #1215203 merged by jenkins-bot:

[mediawiki/core@master] Followup Ie40b9e59a4: Fortify unified metrics method

https://gerrit.wikimedia.org/r/1215203

Change #1215259 had a related patch set uploaded (by Jforrester; author: Mooeypoo):

[mediawiki/core@wmf/1.46.0-wmf.5] Followup Ie40b9e59a4: Fortify unified metrics method

https://gerrit.wikimedia.org/r/1215259

Change #1215259 merged by jenkins-bot:

[mediawiki/core@wmf/1.46.0-wmf.5] Followup Ie40b9e59a4: Fortify unified metrics method

https://gerrit.wikimedia.org/r/1215259

Mentioned in SAL (#wikimedia-operations) [2025-12-04T21:37:48Z] <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1215259|Followup Ie40b9e59a4: Fortify unified metrics method (T411793)]]

Mentioned in SAL (#wikimedia-operations) [2025-12-04T21:40:02Z] <jforrester@deploy2002> jforrester: Backport for [[gerrit:1215259|Followup Ie40b9e59a4: Fortify unified metrics method (T411793)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-12-04T21:45:04Z] <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1215259|Followup Ie40b9e59a4: Fortify unified metrics method (T411793)]] (duration: 07m 16s)

Jdforrester-WMF subscribed.

Backported and noise is gone.