We found in the Grimoire stats for Maniphest that the number of closed tickets per month up to 2014 was 0, which obviously is wrong, since Bugzilla tickets were migrated to Maniphest.
The problem seems to be due to the migration process. Apparently, tickets closed in Bugzilla were migrated to Maniphest as closed tasks, but without a related transaction for the status change. So the table phabricator_maniphest.maniphest_task has the column closedEpoch as NULL, and the trouble is that for calculating closed tickets per month, and the backlogs in the past, in Grimoire we use "closing events", which must have a date.
See for example these two, closed but without closing event:
Compared to these two (look for "closed this task as Resolved"):
I wonder if the missing closing events could be migrated to Maniphest. If that is not possible, we will look for some workaround in Grimoire, but the stats for before 2014 will probably render void.
Bugzilla had about 7500 duplicates, with this distribution over time:
✅ 1. Data scraping
In the next days (2023-07) we will scrape Bugzilla history pages to generate something like this in CSV:
Here the full raw data-source in JSON:
The goal of the above dataset is to capture all the "meaningful status changes" and have a CSV with the "most meaningful status change".
✅ 2. Data matching
Please add here some instructions about how to do the match BugZillaID → PhabricatorID ✨
✅ 3. Create Update queries
The goal here is to execute a lot of UPDATE queries against the phabricator_maniphest.maniphest_task database table having lot of expected Phabricator numerical IDs and close dates. Example:
UPDATE phabricator_maniphest.maniphest_task SET closedEpoch = :epoch: WHERE closedEpoch IS NULL AND id = :expected_phab_id:
4. Epic Database fiddling (help needed)
Somebody with database access to Phabricator production could execute these, to fix the problem:
Preview of first rows:
-- This script is intended to be executed using this database: phabricator_maniphest -- skipping BugZilla ID 1 UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1103779357 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2002; -- in date 2004-12-23 05:22:37 UTC BugZilla ID 2 closed by user colin.pitts with status FIXED UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1092440778 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2003; -- in date 2004-08-13 23:46:18 UTC BugZilla ID 3 closed by user brion with status FIXED -- skipping BugZilla ID 5 -- skipping BugZilla ID 6 UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1368229779 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2007; -- in date 2013-05-10 23:49:39 UTC BugZilla ID 7 closed by user gwicke with status WONTFIX UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1259607436 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2008; -- in date 2009-11-30 18:57:16 UTC BugZilla ID 8 closed by user ayg with status FIXED UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1132381430 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2009; -- in date 2005-11-19 06:23:50 UTC BugZilla ID 9 closed by user brion with status FIXED UPDATE maniphest_task USE INDEX (PRIMARY) SET closedEpoch = 1182970947 WHERE closedEpoch IS NULL AND status NOT IN( 'open', 'stalled', 'duplicate', 'progress' ) AND id = 2010; -- in date 2007-06-27 19:02:27 UTC BugZilla ID 10 closed by user wmbugs with status WONTFIX ...
The above is just a preview. The file has a total of 56 000+ updates, all of them share the "same structure", they "just" touch closedEpoch if NULL and if not open 🌈