Deprecation warnings from MediaWiki are currently not seen by Scap canary-checker, Scap local prepromote checks, or other reguluar production monitoring (e.g. Icgina MW-error alerts).
This means it is relatively easy for these to slip into production unnoticed until group2 where they could overwhelm/saturate Logstash capacity and only trigger alerts there, which is way too late. Or if they are only triggered by a subset of requests, then it's quite likely it would not bring down Logstash until there are multiple such regressions accumulated.
- We already catch deprecation warnings in our CI pipeline if they are triggered by PHP code covered by tests of any kind (PHPUnit, Selenium etc.). Our CI fails the build for deprecation warnings the same way it would for any other PHP notice, warning, error, or fatal exception.
- The Logstash channel:deprecation channel is nearly always empty. This means we have a clean slate and a signal-to-noise ratio at or approaching infinity considering hard-deprecations in production are extremely likely to be regressions and generally relatively easy to mitigate.
Hard deprecations emitted in production should be noticed by:
- Scap's prepromote check, which asserts the PHP stderr from local mwscript invocation to be empty.
- Scaps' canary checker, which currenly monitors the Logstash query for type:mediawiki channel:(exception OR error).
- The mediawiki-new-errors Kibana dashboard.
Basically we can either update a bunch of queries, or we can do what we already do in PHPUnit and what PHP natively already does for its own deprecations, which is to emit them as PHP Warnings. This means they naturally end up in channel:error and in the CLI stderr, similar to messages logged by wfLogWarning and messages from PHP natively (e.g. "PHP Notice: Undefined variable").
I suggest the latter.