Page MenuHomePhabricator

Add mwversion to php7-fatal-error.php logstash message
Open, LowPublic

Description

When triaging errors for a new deployment it helps to filter out previous releases while in the earlier phases of a progressive rollout. This way you start empty, and can triage things as they come in (either filtering them out, or reporting them on phab etc.).

The issue is that right now this means fatal errors are omitted as mwversion:<something> will filter out events both from the old and new deployments as they don't carry this field.

This is technically not something we can know from the post-shutdown stage in the php-wmerrors handler, but I think we can do a best-effort approach here by plucking it out of the entry point's file path and fallback to doing what we do today (which is to not report it).

Perhaps in a future for multiple php-fpm instances and/or containers, this would be trivial if e.g. the deployed version is identified through an environment variable, but we're not there (yet).

The php-wmerors handler is maintained through puppet.git.

Event Timeline

Krinkle created this task.May 27 2020, 6:03 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 27 2020, 6:03 PM

The current workaround is to use the following query:

This
mwversion:"1.35.0-wmf.32" OR exception.file:"1.35.0-wmf.32"

instead of:

(Not this)
mwversion:"1.35.0-wmf.32"

However, where the latter can be done through a clickable preset from the piechart in Kibana, the former cannot (and it is not possible to add a top-level OR to a filter bubble in Kibana). So this requires both parts to be entered manually.

Krinkle claimed this task.Jul 14 2020, 4:40 PM
Krinkle triaged this task as Low priority.
Krinkle updated the task description. (Show Details)
fgiunchedi moved this task from Inbox to Radar on the observability board.Jul 20 2020, 1:11 PM