Page MenuHomePhabricator

Error when provisioning mediawiki-vagrant: "The UdpSocket to 127.0.0.1:10514 has been closed and can not be written to anymore"
Open, MediumPublic

Description

This error shows up when provisioning the elk role:

==> default: Notice: /Stage[main]/Role::Ores/Mediawiki::Maintenance[check ORES model versions]/Exec[check ORES model versions]/returns: PHP Fatal error:  Uncaught exception 'LogicException' with message 'The UdpSocket to 127.0.0.1:10514 has been closed and can not be written to anymore' in /vagrant/mediawiki/vendor/monolog/monolog/src/Monolog/Handler/SyslogUdp/UdpSocket.php:45

A similar exception is shown on every page view too.

This is caused by LBFactory::__destruct (indirectly) logging messages after UdpSocket::__destruct was called and thus the socket does not exist anymore.

We should either make sure that all long handlers swallow errors when called after they have been destroyed, or that no classes try to log from their destructor.

Event Timeline

The error is from LBFactory::__destruct calling LoadBalancer::closeAll which does some debug logging, and at that point UdpSocket::$socket is null because the object's own destructor has already been called.

Change 323360 had a related patch set uploaded (by Gergő Tisza):
Wrap SyslogHandler in WhatFailureGroupHandler in elk role

https://gerrit.wikimedia.org/r/323360

Change 323360 merged by jenkins-bot:
Wrap SyslogHandler in WhatFailureGroupHandler in elk role

https://gerrit.wikimedia.org/r/323360

Tgr added a subscriber: bd808.

Per IRC discussion this was fixed in vagrant by wrapping in WhatFailureGroupHandler like WMF production does. Third parties are on their own I guess.

@bd808 is there a way to tell if the exception is being thrown in production? Maybe the perf team is interested in tracking it down - even if it's caught, throwing exceptions is not that cheap.

@bd808 is there a way to tell if the exception is being thrown in production? Maybe the perf team is interested in tracking it down - even if it's caught, throwing exceptions is not that cheap.

The only way to tell would be either disabling our use of WhatFailureGroupHandler or subclassing it to add some kind of counter that could be monitored. Based on your local experience I'd be pretty comfortable saying that it is throwing exceptions. Logging during the destructor phase of the PHP lifecycle is a difficult thing to get right.

Adding Performance-Team per previous comment. Feel free the close as resolved if you are not interested, the original problem is already fixed.

There is a similar issue on monolog #860 about catching the exception thrown by the handler and/or re-routing the original message to another handler.

Krinkle triaged this task as Medium priority.May 26 2017, 5:01 PM
Krinkle renamed this task from The UdpSocket to 127.0.0.1:10514 has been closed and can not be written to anymore to Error when provisioning mediawiki-vagrant: "The UdpSocket to 127.0.0.1:10514 has been closed and can not be written to anymore".Sep 11 2019, 6:44 PM