Maniphest Reports break when you have 75k tasks
Closed, ResolvedPublic

Description

Upstream task: https://secure.phabricator.com/T6656

This was a problem that we saw during the Bugzilla migration weekend. Reporting it here. We should report this upstream. I think Chase or Mukunda had some IRC chat with the maintainers, but no task submitted?

If you go to https://phabricator.wikimedia.org/maniphest/report/ you get

Request aborted by debug time limit after 30 seconds.

STACK TRACE
ManiphestTask.php:153 PhabricatorStartup->onDebugTick()
ManiphestTaskQuery.php:349 ManiphestTask->attachProjectPHIDs()
PhabricatorPolicyAwareQuery.php:260 ManiphestTaskQuery->didFilterPage()
ManiphestReportController.php:409 PhabricatorPolicyAwareQuery->execute()
ManiphestReportController.php:46 ManiphestReportController->renderOpenTasks()
AphrontController.php:33 ManiphestReportController->processRequest()
index.php:103 AphrontController->handleRequest()
Qgil created this task.Nov 25 2014, 8:17 AM
Qgil updated the task description. (Show Details)
Qgil raised the priority of this task from to Normal.
Qgil changed Security from none to None.
Qgil added a subscriber: Qgil.

@mmodell put that in https://secure.phabricator.com/P1427 but not sure if there's a corresponding upstream ticket yet?

Qgil updated the task description. (Show Details)Nov 27 2014, 8:39 AM
Qgil moved this task from Backlog to Wikimedia requests on the Phabricator (Upstream) board.
Qgil lowered the priority of this task from Normal to Lowest.Dec 2 2014, 9:06 AM

Evan's reply upstream:

https://secure.phabricator.com/T4171 (Building reporting and data systems) is the real solution to this (...)

Alright, this means a total rewrite. And this means that we are not going to have a short term solution unless someone finds a hackish solution to the current code. There is some discussion in https://secure.phabricator.com/T6656.

Meanwhile, I will set realistic priority here. For short term Wikimedia Phabricator reports, see T1003: Monthly report of total / active Phabricator users.

Gilles added a subscriber: Gilles.Dec 2 2014, 9:27 AM

It's probably not a good idea to have a page that hangs for 30 seconds displayed so prominently. The link should be removed/hidden from the maniphest page until a solution is found.

Qgil added a subscriber: chasemp.Dec 2 2014, 2:22 PM

Patch landed upstream:

With the patch, the reports page generates in ~500ms now on this install with 6.5K tasks, so I'd expect it to take ~6s to generate on WMF with 75K tasks if everything scales linearly (which it hopefully should). Still not great, but way better than totally broken. Let me know if you actually see that sort of runtime?

Great catch on the getDestinationPHIDs() issue, in any case.

Qgil closed this task as Resolved.Dec 13 2014, 10:13 AM
Qgil claimed this task.

This was fixed after our last upgrade. Now loading https://phabricator.wikimedia.org/maniphest/report/ takes some seconds, but it loads reliably.

Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptMay 23 2016, 6:05 PM