| • thcipriani | |
| Dec 2 2020, 7:24 PM |
| F33933009: Screenshot 2020-12-03 at 01.22.38.png | |
| Dec 3 2020, 1:22 AM |
In triaging the train we noticed that memory allocation errors were being shown despite filters for that message.
This is due to a change in how monolog reports messages to logstash. Previously, there was an exception object with:
Now there is no exception object, but there is only a message field. We're missing the stack trace and the exception.message.
I guess the easiest is probably to revert the monolog update in master and 1.36.0-wmf.20 ?
I'm guessing the crux of the issue is probably the fix I put in for the Removed Logstash V0 support change in to prevent the bump to V1 - T247675: Stop overriding LogstashFormatter format is the task for trying to clean that up eventually
https://github.com/Seldaek/monolog/blob/1.x/src/Monolog/Formatter/LogstashFormatter.php#L87-L128
https://github.com/Seldaek/monolog/blob/1.x/src/Monolog/Formatter/LogstashFormatter.php#L130-L165
https://github.com/Seldaek/monolog/blob/2.0.2/src/Monolog/Formatter/LogstashFormatter.php#L64-L100
Looking again at the code I copied out... It seems the @ disappeared
Let me have a poke
Change 644910 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] Correctly import LogstashFormatter::v0 from monolog/monolog 1.25.3
Change 644915 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/vendor@master] Revert monolog/monolog (2.0.0 => 1.25.3)
Change 644917 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@master] Revert "Updating monolog/monolog (1.25.3 => 2.0.2)"
I think my latest PS to https://gerrit.wikimedia.org/r/c/mediawiki/core/+/644910 (tests pass, woo!) should get us back in a good state
Taking a break to get some food, then I'll come back and finish off
Change 644822 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@wmf/1.36.0-wmf.20] Correctly forwardport LogstashFormatter from monolog/monolog 1.25.3
Change 644822 merged by jenkins-bot:
[mediawiki/core@wmf/1.36.0-wmf.20] Correctly forwardport LogstashFormatter from monolog/monolog 1.25.3
Mentioned in SAL (#wikimedia-operations) [2020-12-02T23:42:03Z] <reedy@deploy1001> Synchronized php-1.36.0-wmf.20/includes/debug/logger/monolog/LogstashFormatter.php: T269286 (duration: 01m 07s)
In triaging the train we noticed that memory allocation errors were being shown despite filters for that message.
They're gone... Still visible on mwlog1001... I think this is fixed
Confirming fixed; I just filed T269314: Argument 5 passed to MediaWiki\Extension\AbuseFilter\Filter\Specs::__construct() must be of the type string, null given, called in /srv/mediawiki/php-1.36.0-wmf.20/extensions/AbuseFilter/includes/FilterLookup.php on line 346 from kibana to Phab from .20
Will unblock parent
Change 644917 abandoned by Reedy:
[mediawiki/core@master] Revert "Updating monolog/monolog (1.25.3 => 2.0.2)"
Reason:
Change 644915 abandoned by Reedy:
[mediawiki/vendor@master] Revert monolog/monolog (2.0.0 => 1.25.3)
Reason:
Change 644910 merged by jenkins-bot:
[mediawiki/core@master] Correctly forwardport LogstashFormatter from monolog/monolog 1.25.3
Change 664702 had a related patch set uploaded (by Reedy; owner: Reedy):
[mediawiki/core@REL1_35] Correctly forwardport LogstashFormatter from monolog/monolog 1.25.3
Change 664702 merged by jenkins-bot:
[mediawiki/core@REL1_35] Correctly forwardport LogstashFormatter from monolog/monolog 1.25.3