Page MenuHomePhabricator

Internal Server Error
Closed, ResolvedPublic

Description

On responding to a ticket (any ticket), the following error is presented in the -Reply- pop up:

The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at webmaster@wikimedia.org to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

Date/Time: 26/7/2016 @ 11:19 UTC
Browser: Firefox 48.0b10
OS: Windows 10 Pro

Event Timeline

pajz triaged this task as High priority.Jul 26 2016, 11:29 AM
pajz added subscribers: akosiaris, Krd, pajz.

Can confirm (tested for several queues, including info-de, and for different response templates, including "empty answer").

TheresNoTime claimed this task.

Transient issue, resolved at 11:29 UTC - might be worth checking out anyway

Re-opening, hadn't seen previous reply - feel free to claim

Indeed, looks like the error has resolved itself. Seems to work fine again. I'll check again in a few minutes.

pajz claimed this task.

Still looks fine. Closing ...

There seems there has been some transient issue in the [Tue Jul 26 11:12:28.018820 2016] to [Tue Jul 26 11:46:39.031559 2016] time window. Logs were not indicative of a definite failure. Mostly lines of the sort

Can't locate object method "new" via package "Net::DNS::DomainName1035" at /usr/lib/x86_64-linux-gnu/perl5/5.20/Net/DNS/Question.pm line 80

Still investigating.

So there was a libnet-dns-perl upgrade on mendelevium that likely caused this. Most likely apache mod_perl children kept a open reference to the old perl module and could not find it after the upgrade. It was fixed by the dying apache children being restarted and reloading the code along with the upgraded perl module. I 'll be monitoring in case it manifests again and orchestrating the perl upgrades on mendelevium a bit better.