https://phabricator.wikimedia.org/project/sprint/burn/483/ really intensive request leads to either 504 or 503.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | None | T107197 Sprint extension doesn't scale to thousands of tasks in a single sprint: burndown page exceeds max execution timeout on visual editor project | |||
| Resolved | • mmodell | T108142 Next Phabricator Upgrade: on 2015-08-26 |
Event Timeline
Comment Actions
Request aborted by debug time limit after 20 seconds. STACK TRACE qsprintf.php:149 PhabricatorStartup->onDebugTick() xsprintf.php:70 xsprintf_query() qsprintf.php:64 xsprintf() PhabricatorApplicationTransactionCommentQuery.php:80 qsprintf() PhabricatorApplicationTransactionCommentQuery.php:61 PhabricatorApplicationTransactionCommentQuery->buildWhereClauseComponents() PhabricatorApplicationTransactionCommentQuery.php:53 PhabricatorApplicationTransactionCommentQuery->buildWhereClause() PhabricatorPolicyAwareQuery.php:227 PhabricatorApplicationTransactionCommentQuery->loadPage() PhabricatorApplicationTransactionQuery.php:78 PhabricatorPolicyAwareQuery->execute() PhabricatorPolicyAwareQuery.php:227 PhabricatorApplicationTransactionQuery->loadPage() SprintQuery.php:163 PhabricatorPolicyAwareQuery->execute() SprintDataView.php:154 SprintQuery->getXactions() SprintDataView.php:47 SprintDataView->setEvents() AphrontView.php:176 SprintDataView->render() render.php:133 AphrontView->producePhutilSafeHTML() render.php:157 phutil_escape_html() render.php:157 phutil_escape_html() render.php:157 phutil_escape_html() render.php:119 phutil_escape_html() AphrontSideNavFilterView.php:312 phutil_tag() AphrontSideNavFilterView.php:206 AphrontSideNavFilterView->renderFlexNav() AphrontView.php:176 AphrontSideNavFilterView->render() render.php:133 AphrontView->producePhutilSafeHTML() render.php:189 phutil_escape_html() PhabricatorBarePageView.php:58 phutil_implode_html() PhabricatorStandardPageView.php:157 PhabricatorBarePageView->willRenderPage() AphrontPageView.php:46 PhabricatorStandardPageView->willRenderPage() PhabricatorController.php:308 AphrontPageView->render() PhabricatorController.php:370 PhabricatorController->buildPageResponse() SprintDataViewController.php:50 PhabricatorController->buildApplicationPage() AphrontApplicationConfiguration.php:226 SprintDataViewController->handleRequest() AphrontApplicationConfiguration.php:140 AphrontApplicationConfiguration->processRequest() index.php:21 AphrontApplicationConfiguration->runHTTPRequest()
Comment Actions
My best guess is that the sprint extension just isn't scalable enough to handle the amount of data for this particular sprint.
Comment Actions
Why is VisualEditor a sprint? Shouldn't a sprint be a time-limited thing, not an ongoing project?
Comment Actions
Why is VisualEditor a sprint? Shouldn't a sprint be a time-limited thing, not an ongoing project?
Because we can't add points to tasks unless we claim it's a sprint.
Comment Actions
Specifically, we want to maintain a backlog, which is currently 1300+ items, that is retrievable and sortable in a query. And we want some items in the backlog, maybe even all of them, to have story points. Open to other suggestions to achieve this outcome.
Comment Actions
How about creating an Upstream (Phabricator (Upstream))-task to add poins without being sprint? Would be one way. But that's not going to "fix" this issue, it's just a workaround the "crash".
Comment Actions
>>> UNRECOVERABLE FATAL ERROR <<< Maximum execution time of 30 seconds exceeded /srv/phab/libphutil/src/symbols/PhutilSymbolLoader.php:372 ┻━┻ ︵ ¯\_(ツ)_/¯ ︵ ┻━┻
Got to love those emoticons in crashlogs :)
Comment Actions
There is an easy fix for this. Just disable the events view. This is a global config option and will affect all sprints, but clearly the loading of thousands of events does not scale for a single page load. This will drastically reduce the load time for the burndown view.
Comment Actions
@Christopher: sadly, disabling events view doesn't seem to solve the problem - it's still timing out.
Comment Actions
Here's the stack with the events view disabled:
STACK TRACE LiskDAO.php:608 PhabricatorStartup->onDebugTick() LiskDAO.php:655 LiskDAO->loadFromArray() PhabricatorApplicationTransactionQuery.php:62 LiskDAO->loadAllFromArray() PhabricatorPolicyAwareQuery.php:227 PhabricatorApplicationTransactionQuery->loadPage() SprintQuery.php:163 PhabricatorPolicyAwareQuery->execute() SprintDataView.php:154 SprintQuery->getXactions() SprintDataView.php:47 SprintDataView->setEvents() AphrontView.php:176 SprintDataView->render() render.php:133 AphrontView->producePhutilSafeHTML() render.php:157 phutil_escape_html() render.php:157 phutil_escape_html() render.php:157 phutil_escape_html() render.php:119 phutil_escape_html() AphrontSideNavFilterView.php:312 phutil_tag() AphrontSideNavFilterView.php:206 AphrontSideNavFilterView->renderFlexNav() AphrontView.php:176 AphrontSideNavFilterView->render() render.php:133 AphrontView->producePhutilSafeHTML() render.php:189 phutil_escape_html() PhabricatorBarePageView.php:58 phutil_implode_html() PhabricatorStandardPageView.php:157 PhabricatorBarePageView->willRenderPage() AphrontPageView.php:46 PhabricatorStandardPageView->willRenderPage() PhabricatorController.php:308 AphrontPageView->render() PhabricatorController.php:370 PhabricatorController->buildPageResponse() SprintDataViewController.php:50 PhabricatorController->buildApplicationPage() AphrontApplicationConfiguration.php:226 SprintDataViewController->handleRequest() AphrontApplicationConfiguration.php:140 AphrontApplicationConfiguration->processRequest() index.php:21 AphrontApplicationConfiguration->runHTTPRequest()
Comment Actions
Yeah this just isn't going to ever work with that much data. It's querying all of the transactions (and comments!) for all of the tasks in the project.
Comment Actions
See https://gerrit.wikimedia.org/r/#/c/231421/
Even though config option disabling the event view was set, the query method setEvents that loads the data was still executing. When I profiled this change, it reduced execution time by nearly 50%.
This could fix the problem.
Comment Actions
Testing change by profiling 911 tasks in one project on phab09:
inclusive wall time on burndown view:
10,010,876 us with events table enabled
2,717,320 us with events table disabled
I am fairly sure that the sprint extension can scale to thousands of tasks in a single project. But it cannot load tens of thousands of event transactions in one page view.
Comment Actions
Still not fixed. (VE has 3866 tasks).
A complication is the function that evaluates and builds a map for task dependencies. I suspect that this adds significant processing time when loading the task table. I do not see an easy fix for this other than to disable the task table. A disable/enable configuration is possible on a per project basis if a new custom field were introduced to store it (and an interface to set it). This development path leads to more complexity, however, and I guess that the goal remains to shift this project management tool to Phragile soon.
Note that Collaboration-Team Backlog loads with 1783 tasks. https://phabricator.wikimedia.org/project/sprint/view/65/. I think that this is approaching the upper task limit.