Page MenuHomePhabricator

ManiphestReportController uses a heavy/slow ManiphestTransaction table query
Closed, ResolvedPublic

Description

https://we.phorge.it/source/phorge/browse/master/src/applications/maniphest/controller/ManiphestReportController.php$87-121 is the only place in upstream which queries the ManiphestTransaction table:

$table = new ManiphestTransaction();
$conn = $table->establishConnection('r');
[...]
$data = queryfx_all(
[...]
$table->getTableName(),

This is getting more and more expensive in our installation so I already had to adjust some other custom downstream code querying this table.[1],[2]

As I wrote in T119376#9833809,

in the long term, I expect the upstream report/chart code to break anyway due to timeouts, given the current database structure.

Upstream code should likely be ported to use something similar to a Materialized View, cf https://dba.stackexchange.com/questions/86790/best-way-to-create-a-materialized-view-in-mysql

Event Timeline

Aklapper triaged this task as Low priority.

As I doubt that the Materialized View approach is feasible (given that the code allows a project parameter and we have thousands of projects), I filed https://we.phorge.it/D25828 in upstream to split handling of legacy data (querying the large ManiphestTransaction table) and modern/synthetic data (NOT querying the ManiphestTransaction table), with a vague followup idea to completely disable the legacy code and thus ignore data from before 2018 (*), for the sake of performance.

(*) Well, certain data. I believe that the modern handling should still be able to create most of pre-2018 data in a way more performant way.

There is also a lot of useless code in that class, wasting CPU cycles: https://we.phorge.it/D25902

I only recently realized that this entire expensive code is completely unneeded and can be removed. Upstream: https://we.phorge.it/D25902

Aklapper changed the task status from Open to Stalled.Mar 6 2025, 10:02 AM
Aklapper moved this task from Backlog to Upstreamed on the Phabricator (Upstream) board.

This issue should now be fixed on phabricator.wikimedia.org after today's software deployment in T404134.