Odd error whilst navigating to https://phabricator.wikimedia.org/project/sprint/burn/1113/
Closed, ResolvedPublic3 Story Points

Description

>>> UNRECOVERABLE FATAL ERROR <<<

Maximum execution time of 30 seconds exceeded

/srv/phab/phabricator/src/infrastructure/query/policy/PhabricatorPolicyAwareQuery.php:0


┻━┻ ︵ ¯\_(ツ)_/¯ ︵ ┻━┻
Jdforrester-WMF updated the task description. (Show Details)
Jdforrester-WMF raised the priority of this task from to Normal.
Jdforrester-WMF added a subscriber: Jdforrester-WMF.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 11 2015, 4:43 PM
Qgil added a subscriber: Qgil.Jun 11 2015, 7:05 PM

I got another error:

Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Please try again in a few minutes.

If you report this error to the Wikimedia System Administrators, please include the details below.
Request: GET http://phabricator.wikimedia.org/project/sprint/burn/1113/, from 10.64.0.171 via cp1043 cp1043 ([10.64.0.171]:80), Varnish XID 2120040637
Forwarded for: 2003:5b:ee5c:46ff:8c3c:769f:70c1:ae51, 10.64.0.171
Error: 503, Service Unavailable at Thu, 11 Jun 2015 19:04:08 GMT

Clickable link for convenience: https://phabricator.wikimedia.org/project/sprint/burn/1113/

My guess: https://phabricator.wikimedia.org/project/sprint/details/1113/ has "Editable By" set to "Custom Policy" (to allow only five users). Hence more complex to check and ending up with a performance issue.

I'm not sure why this restriction was done (@Jdforrester-WMF might remember); what the actual intention of it is (and if the intention is related to the actual restriction ;-), and I'm curious if using the default policy would make this error vanish.

This comment was removed by Christopher.

The problem is a direct consequence of the new method that was introduced with the latest commit https://gerrit.wikimedia.org/r/#/c/217183/.

In looking at the query analysis plan, it became totally obvious to me that it was retrieving all project column transactions before filtering them for scope!

I just pushed the fix here: https://gerrit.wikimedia.org/r/#/c/217711/2. It adds a where to the transaction query for tasks in the sprint before it executes (not after...).

I really think that this should be hotfixed if possible.

Christopher set Security to None.
Christopher edited a custom field.

@mmodell Can you hotfix this 2 line patch https://gerrit.wikimedia.org/r/#/c/217711/2? Until this is merged, burndown charts are effectively broken.

Qgil raised the priority of this task from Normal to Unbreak Now!.Jun 12 2015, 6:54 AM
Qgil added a project: acl*operations-team.
Qgil added subscribers: chasemp, Dzahn.

@mmodell Can you hotfix this 2 line patch https://gerrit.wikimedia.org/r/#/c/217711/2? Until this is merged, burndown charts are effectively broken.

Escalating, just in case.

Change 217845 had a related patch set uploaded (by 20after4):
Fix T102142 - hotfix for phabricator Sprint extension.

https://gerrit.wikimedia.org/r/217845

mmodell added a comment.EditedJun 12 2015, 1:59 PM

I already forced the pull on the phab server, but the patch still needs to be merged into operations/puppet to make it official. https://phabricator.wikimedia.org/project/sprint/burn/1113/ loads now, still slowly but it does eventually load.

Change 217845 merged by Rush:
Fix T102142 - hotfix for phabricator Sprint extension.

https://gerrit.wikimedia.org/r/217845

I already forced the pull on the phab server, but the patch still needs to be merged into operations/puppet to make it official. https://phabricator.wikimedia.org/project/sprint/burn/1113/ loads now, still slowly but it does eventually load.

done

Thanks everybody for the quick fix!
Confirming that https://phabricator.wikimedia.org/project/sprint/burn/1113/ works for me now.

Restricted Application added a subscriber: scfc. · View Herald TranscriptJun 25 2015, 9:54 AM