Page MenuHomePhabricator

Bug: Trigger caused db locks
Closed, ResolvedPublic

Description

figure out trigger crash

Details

Related Gerrit Patches:
wikimedia/fundraising/crm : masterre-add date of largest contribution
wikimedia/fundraising/crm : masterUpdate backfill command
wikimedia/fundraising/crm : deploymentUpdated triggers
wikimedia/fundraising/crm : deploymentExclude Refund contributions from calculations.
wikimedia/fundraising/crm : masterExclude Refund contributions from calculations.
wikimedia/fundraising/crm : masterWMF donor only triggers file
wikimedia/fundraising/crm : masterCommit updated triggers
wikimedia/fundraising/crm : masterRemove largest_donation_date from calcs

Related Objects

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 29 2019, 8:28 PM

Change 526286 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Remove largest_donation_date from calcs

https://gerrit.wikimedia.org/r/526286

Change 526286 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Remove largest_donation_date from calcs

https://gerrit.wikimedia.org/r/526286

Change 526298 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Commit updated triggers

https://gerrit.wikimedia.org/r/526298

Change 526299 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] WMF donor only triggers file

https://gerrit.wikimedia.org/r/526299

Change 526298 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Commit updated triggers

https://gerrit.wikimedia.org/r/526298

Change 526299 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] WMF donor only triggers file

https://gerrit.wikimedia.org/r/526299

@Jgreen on live there is now a trigger file sites/all/modules/wmf_civicrm/scripts/contribution_triggers.mysql which has the triggers minus the ones I think most likely to be problematic - if you can reload that

@Jgreen on live there is now a trigger file sites/all/modules/wmf_civicrm/scripts/contribution_triggers.mysql which has the triggers minus the ones I think most likely to be problematic - if you can reload that

Done!

OK - we have had one donation come in since they were re-enabled & it worked (short the date_of_largest_donation field which is currently excluded) - I'll work on a backfill now

Back in doing for the backfill script

Change 526798 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Update backfill command

https://gerrit.wikimedia.org/r/526798

OK I'm getting good results on staging even updating 150k contacts at once - so struggling to explain the hang. Still I have got some explain results to dig into running it on a large batch

+------+-------------+------------+--------+------------------------------------+------------------------------------+---------+-----------------------+---------+--------------------------------------------------------+

idselect_typetabletypepossible_keyskeykey_lenrefrowsExtra
1PRIMARY<derived2>ALLNULLNULLNULLNULL1449532Using temporary; Using filesort
1PRIMARYlatestrefFK_civicrm_contribution_contact_idFK_civicrm_contribution_contact_id4totals.contact_id1Using where
1PRIMARYxeq_refunique_entity_idunique_entity_id4dev_civicrm.latest.id1Using where
1PRIMARYearliestrefFK_civicrm_contribution_contact_idFK_civicrm_contribution_contact_id4totals.contact_id1Using where
1PRIMARYlargestrefFK_civicrm_contribution_contact_idFK_civicrm_contribution_contact_id4totals.contact_id1Using where
2DERIVEDcrangeFK_civicrm_contribution_contact_idFK_civicrm_contribution_contact_id4NULL1449532Using index condition; Using temporary; Using filesort

+------+-------------+-

Eileenmcnaughton added a comment.EditedJul 31 2019, 11:53 PM

OK - so the number of rows is double there - ie

select count(*) FROM civicrm_contribution WHERE contact_id > 1000000 AND contact_id <= 1300000;
+----------+

count(*)

+----------+

765702
EXPLAIN  INSERT INTO wmf_donor (
    #### Begin change from stored trigger
    # entity_id,
    #### End change from stored trigger
    
     last_donation_currency, last_donation_amount, last_donation_usd,
         first_donation_usd,
          ### Begin re-include largest date
         date_of_largest_donation,
         ### End re-include largest date
     
         #### Begin change from stored trigger
         entity_id,
         #### End change from stored trigger
     
         largest_donation, endowment_largest_donation, lifetime_including_endowment,
         lifetime_usd_total, endowment_lifetime_usd_total,
         last_donation_date, endowment_last_donation_date, first_donation_date,
         endowment_first_donation_date, number_donations,
         endowment_number_donations, total_2006_2007, total_2006, total_2007_2008, total_2007, total_2008_2009, total_2008, total_2009_2010, total_2009, total_2010_2011, total_2010, total_2011_2012, total_2011, total_2012_2013, total_2012, total_2013_2014, total_2013, total_2014_2015, total_2014, total_2015_2016, total_2015, total_2016_2017, total_2016, total_2017_2018, total_2017, change_2017_2018, total_2018_2019, total_2018, endowment_total_2018_2019, endowment_total_2018, change_2018_2019, total_2019_2020, total_2019, endowment_total_2019_2020, endowment_total_2019, change_2019_2020, total_2020_2021, total_2020, endowment_total_2020_2021, endowment_total_2020, change_2020_2021
       )
     
         SELECT
           # note that all totals are calculated in the totals subquery and only join values
           # in the main query. This is important as the left joins could catch more than one donation at
           # exactly the same time or more than one max donation amount & throw out the totals.

    ->       #### Begin change from stored trigger
    ->       # OLD.contact_id as entity_id,
    ->       #### End change from stored trigger
    ->       COALESCE(x.original_currency, latest.currency) as last_donation_currency,
    ->       COALESCE(x.original_amount, latest.total_amount, 0) as last_donation_amount,
    ->       COALESCE(latest.total_amount, 0) as last_donation_usd,
    ->       COALESCE(earliest.total_amount, 0) as first_donation_usd,
    ->       ### Begin re-include largest date
    ->       largest.receive_date as date_of_largest_donation,
    ->       ### End re-include largest date
    ->       totals.*
    -> 
    ->     FROM (
    ->       SELECT
    ->         #### Begin change from stored trigger
    ->         c.contact_id,
    ->         #### End change from stored trigger
        MAX(IF(financial_type_id <> 26, COALESCE(total_amount, 0), 0)) AS largest_donation,
      MAX(IF(financial_type_id = 26, COALESCE(total_amount, 0), 0)) AS endowment_largest_donation,
      SUM(COALESCE(total_amount, 0)) AS lifetime_including_endowment,
    ->         SUM(IF(financial_type_id <> 26, COALESCE(total_amount, 0), 0)) AS lifetime_usd_total,
    ->         SUM(IF(financial_type_id = 26, COALESCE(total_amount, 0), 0)) AS endowment_lifetime_usd_total,
    ->         MAX(IF(financial_type_id <> 26 AND total_amount > 0, receive_date, NULL)) AS last_donation_date,
    ->         MAX(IF(financial_type_id = 26 AND total_amount > 0, receive_date, NULL)) AS endowment_last_donation_date,
    ->         MIN(IF(financial_type_id <> 26 AND total_amount, receive_date, NULL)) AS first_donation_date,
    ->         MIN(IF(financial_type_id = 26 AND total_amount > 0, receive_date, NULL)) AS endowment_first_donation_date,
    ->         COUNT(IF(financial_type_id <> 26 AND total_amount > 0, receive_date, NULL)) AS number_donations,
    ->         COUNT(IF(financial_type_id = 26 AND total_amount > 0, receive_date, NULL)) AS endowment_number_donations,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2006-07-01' AND '2007-06-30 23:59:59', c.total_amount, 0),0)) as total_2006_2007,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2006-01-01' AND '2006-12-31 23:59:59', c.total_amount, 0),0)) as total_2006,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2007-07-01' AND '2008-06-30 23:59:59', c.total_amount, 0),0)) as total_2007_2008,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2007-01-01' AND '2007-12-31 23:59:59', c.total_amount, 0),0)) as total_2007,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2008-07-01' AND '2009-06-30 23:59:59', c.total_amount, 0),0)) as total_2008_2009,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2008-01-01' AND '2008-12-31 23:59:59', c.total_amount, 0),0)) as total_2008,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2009-07-01' AND '2010-06-30 23:59:59', c.total_amount, 0),0)) as total_2009_2010,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2009-01-01' AND '2009-12-31 23:59:59', c.total_amount, 0),0)) as total_2009,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2010-07-01' AND '2011-06-30 23:59:59', c.total_amount, 0),0)) as total_2010_2011,
    ->         SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2010-01-01' AND '2010-12-31 23:59:59', c.total_amount, 0),0)) as total_2010,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2011-07-01' AND '2012-06-30 23:59:59', c.total_amount, 0),0)) as total_2011_2012,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2011-01-01' AND '2011-12-31 23:59:59', c.total_amount, 0),0)) as total_2011,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2012-07-01' AND '2013-06-30 23:59:59', c.total_amount, 0),0)) as total_2012_2013,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2012-01-01' AND '2012-12-31 23:59:59', c.total_amount, 0),0)) as total_2012,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2013-07-01' AND '2014-06-30 23:59:59', c.total_amount, 0),0)) as total_2013_2014,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2013-01-01' AND '2013-12-31 23:59:59', c.total_amount, 0),0)) as total_2013,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2014-07-01' AND '2015-06-30 23:59:59', c.total_amount, 0),0)) as total_2014_2015,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2014-01-01' AND '2014-12-31 23:59:59', c.total_amount, 0),0)) as total_2014,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2015-07-01' AND '2016-06-30 23:59:59', c.total_amount, 0),0)) as total_2015_2016,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2015-01-01' AND '2015-12-31 23:59:59', c.total_amount, 0),0)) as total_2015,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2016-07-01' AND '2017-06-30 23:59:59', c.total_amount, 0),0)) as total_2016_2017,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2016-01-01' AND '2016-12-31 23:59:59', c.total_amount, 0),0)) as total_2016,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2017-07-01' AND '2018-06-30 23:59:59', c.total_amount, 0),0)) as total_2017_2018,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2017-01-01' AND '2017-12-31 23:59:59', c.total_amount, 0),0)) as total_2017,
    ->           SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2018-01-01' AND '2018-12-31 23:59:59', c.total_amount, 0),0))
    ->           - SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2017-01-01' AND '2017-12-31 23:59:59', c.total_amount, 0),0))
    ->            as change_2017_2018,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2018-07-01' AND '2019-06-30 23:59:59', c.total_amount, 0),0)) as total_2018_2019,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2018-01-01' AND '2018-12-31 23:59:59', c.total_amount, 0),0)) as total_2018,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2018-07-01' AND '2019-06-30 23:59:59', c.total_amount, 0),0)) as endowment_total_2018_2019,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2018-01-01' AND '2018-12-31 23:59:59', c.total_amount, 0),0)) as endowment_total_2018,
    ->           SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2019-01-01' AND '2019-12-31 23:59:59', c.total_amount, 0),0))
    ->           - SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2018-01-01' AND '2018-12-31 23:59:59', c.total_amount, 0),0))
    ->            as change_2018_2019,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2019-07-01' AND '2020-06-30 23:59:59', c.total_amount, 0),0)) as total_2019_2020,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2019-01-01' AND '2019-12-31 23:59:59', c.total_amount, 0),0)) as total_2019,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2019-07-01' AND '2020-06-30 23:59:59', c.total_amount, 0),0)) as endowment_total_2019_2020,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2019-01-01' AND '2019-12-31 23:59:59', c.total_amount, 0),0)) as endowment_total_2019,
    ->           SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2020-01-01' AND '2020-12-31 23:59:59', c.total_amount, 0),0))
    ->           - SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2019-01-01' AND '2019-12-31 23:59:59', c.total_amount, 0),0))
    ->            as change_2019_2020,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2020-07-01' AND '2021-06-30 23:59:59', c.total_amount, 0),0)) as total_2020_2021,SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2020-01-01' AND '2020-12-31 23:59:59', c.total_amount, 0),0)) as total_2020,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2020-07-01' AND '2021-06-30 23:59:59', c.total_amount, 0),0)) as endowment_total_2020_2021,SUM(COALESCE(IF(financial_type_id = 26 AND receive_date BETWEEN '2020-01-01' AND '2020-12-31 23:59:59', c.total_amount, 0),0)) as endowment_total_2020,
    ->           SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2021-01-01' AND '2021-12-31 23:59:59', c.total_amount, 0),0))
    ->           - SUM(COALESCE(IF(financial_type_id <> 26 AND receive_date BETWEEN '2020-01-01' AND '2020-12-31 23:59:59', c.total_amount, 0),0))
    ->            as change_2020_2021
    ->       FROM civicrm_contribution c
    ->       USE INDEX(FK_civicrm_contribution_contact_id)
    -> 
    ->       #### Begin change from stored trigger
    ->       # WHERE contact_id = OLD.contact_id
    ->       WHERE contact_id > 1000000 AND contact_id <= 1300000
    ->       GROUP BY contact_id
    ->       #### End change from stored trigger
    -> 
    ->      AND contribution_status_id = 1
    ->     ) as totals
    ->   LEFT JOIN civicrm_contribution latest
    ->     USE INDEX(FK_civicrm_contribution_contact_id)
    -> 
    ->     #### Begin change from stored trigger
    ->     #ON latest.contact_id = OLD.contact_id
    ->     ON latest.contact_id = totals.contact_id
    ->     #### End change from stored trigger
    ->     
    -> AND latest.receive_date = totals.last_donation_date
    -> AND latest.contribution_status_id = 1
    -> AND latest.total_amount > 0
    -> AND (latest.trxn_id NOT LIKE 'RFD %' OR latest.trxn_id IS NULL)
    ->     AND latest.financial_type_id <> 26
    ->   LEFT JOIN wmf_contribution_extra x ON x.entity_id = latest.id
    -> 
    ->   LEFT JOIN civicrm_contribution earliest
    ->     USE INDEX(FK_civicrm_contribution_contact_id)
    ->     #### Begin change from stored trigger
    ->     # ON earliest.contact_id = OLD.contact_id
    ->     ON earliest.contact_id = totals.contact_id
    ->     #### End change from stored trigger
    -> 
    -> AND earliest.receive_date = totals.first_donation_date
    -> AND earliest.contribution_status_id = 1
    -> AND earliest.total_amount > 0
    -> AND (earliest.trxn_id NOT LIKE 'RFD %' OR earliest.trxn_id IS NULL)
    -> 
    ->   ### Begin re-include largest date
    ->     LEFT JOIN civicrm_contribution largest
    ->     USE INDEX(FK_civicrm_contribution_contact_id)
    ->     ### begin alter for backfill
    ->     ON largest.contact_id = totals.contact_id
    ->     ### end alter for backfill
    ->     AND largest.total_amount = totals.largest_donation
    ->     AND largest.contribution_status_id = 1
    ->     AND largest.total_amount > 0
    ->     AND (largest.trxn_id NOT LIKE 'RFD %' OR largest.trxn_id IS NULL)
    ->   ORDER BY largest.receive_date DESC
    ->   ### End re-include largest date
    -> 
    ->   LIMIT 1
    -> 
    ->   ON DUPLICATE KEY UPDATE
    ->     last_donation_currency = VALUES(last_donation_currency),
    ->     last_donation_amount = VALUES(last_donation_amount),
    ->     last_donation_usd = VALUES(last_donation_usd),
    ->     first_donation_usd = VALUES(first_donation_usd),
    ->     largest_donation = VALUES(largest_donation),
    ->     ### Begin -re-intro largest-date
    ->     date_of_largest_donation = VALUES(date_of_largest_donation),
    ->     ### End -re-intro largest-date
    ->     lifetime_usd_total = VALUES(lifetime_usd_total),
    ->     last_donation_date = VALUES(last_donation_date),
    ->     first_donation_date = VALUES(first_donation_date),
    ->     number_donations = VALUES(number_donations),
    ->     endowment_largest_donation = VALUES(endowment_largest_donation),
    ->     lifetime_including_endowment = VALUES(lifetime_including_endowment),
    ->     endowment_lifetime_usd_total = VALUES(endowment_lifetime_usd_total),
    ->     endowment_last_donation_date = VALUES(endowment_last_donation_date),
    ->     endowment_first_donation_date = VALUES(endowment_first_donation_date),
    ->     endowment_number_donations = VALUES(endowment_number_donations),
    ->     total_2006_2007 = VALUES(total_2006_2007),total_2006 = VALUES(total_2006),total_2007_2008 = VALUES(total_2007_2008),total_2007 = VALUES(total_2007),total_2008_2009 = VALUES(total_2008_2009),total_2008 = VALUES(total_2008),total_2009_2010 = VALUES(total_2009_2010),total_2009 = VALUES(total_2009),total_2010_2011 = VALUES(total_2010_2011),total_2010 = VALUES(total_2010),total_2011_2012 = VALUES(total_2011_2012),total_2011 = VALUES(total_2011),total_2012_2013 = VALUES(total_2012_2013),total_2012 = VALUES(total_2012),total_2013_2014 = VALUES(total_2013_2014),total_2013 = VALUES(total_2013),total_2014_2015 = VALUES(total_2014_2015),total_2014 = VALUES(total_2014),total_2015_2016 = VALUES(total_2015_2016),total_2015 = VALUES(total_2015),total_2016_2017 = VALUES(total_2016_2017),total_2016 = VALUES(total_2016),total_2017_2018 = VALUES(total_2017_2018),total_2017 = VALUES(total_2017),change_2017_2018 = VALUES(change_2017_2018),total_2018_2019 = VALUES(total_2018_2019),total_2018 = VALUES(total_2018),endowment_total_2018_2019 = VALUES(endowment_total_2018_2019),endowment_total_2018 = VALUES(endowment_total_2018),change_2018_2019 = VALUES(change_2018_2019),total_2019_2020 = VALUES(total_2019_2020),total_2019 = VALUES(total_2019),endowment_total_2019_2020 = VALUES(endowment_total_2019_2020),endowment_total_2019 = VALUES(endowment_total_2019),change_2019_2020 = VALUES(change_2019_2020),total_2020_2021 = VALUES(total_2020_2021),total_2020 = VALUES(total_2020),endowment_total_2020_2021 = VALUES(endowment_total_2020_2021),endowment_total_2020 = VALUES(endowment_total_2020),change_2020_2021 = VALUES(change_2020_2021);
mepps renamed this task from Figure out crashing state to Bug: Trigger caused db locks.Aug 2 2019, 2:35 PM

Change 527706 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Exclude Refund contributions from calculations.

https://gerrit.wikimedia.org/r/527706

Change 527706 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Exclude Refund contributions from calculations.

https://gerrit.wikimedia.org/r/527706

Change 528295 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@deployment] Exclude Refund contributions from calculations.

https://gerrit.wikimedia.org/r/528295

Change 528295 merged by jenkins-bot:
[wikimedia/fundraising/crm@deployment] Exclude Refund contributions from calculations.

https://gerrit.wikimedia.org/r/528295

Change 528310 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] re-add date of largest contribution

https://gerrit.wikimedia.org/r/528310

Change 528594 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@deployment] Updated triggers

https://gerrit.wikimedia.org/r/528594

Change 528594 merged by jenkins-bot:
[wikimedia/fundraising/crm@deployment] Updated triggers

https://gerrit.wikimedia.org/r/528594

@Jgreen we have made a minor update to the triggers but I need you to reload them now - the change is per https://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528594/1/sites/all/modules/wmf_civicrm/scripts/triggers.mysql

and the updated triggers are on prod at sites/all/modules/wmf_civicrm/scripts/triggers.mysql

The change just excludes adjustments from the totals (ie. if you are refunded your $3 donation but due to currency changes we still retain 13 cents the 13 cents is now excluded. It is preparatory to rerunning the script to fill the data missed while triggers were off

Change 526798 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Update backfill command

https://gerrit.wikimedia.org/r/526798

We have discussed turning this back on tomorrow. I have done testing on staging for performance with 4 variants per the table below - I ran them several times each since they vary a little. The 2 things I tested were

  1. making the query compatible with Full Group By mode (which is not enabled on live but is best practice)
  2. re-enabling the field.

I'm a little on the fence as to whether the FGB version of the query was marginally slower - since the granularity is really 'did it take 66 or 67 seconds to import 500 rows' - I suspect the difference is within the margin of error - results below

File nameIs full group by mode compatibleHas date of latest contribution fieldPatchtime takennumber per minute
1-civicrm_contribution_trigger_as_running_on_live.mysqlNoNoHEAD66 second(s)455 per minute
1-civicrm_contribution_trigger_as_running_on_live.mysqlNoNoHEAD66 second(s)455 per minute.
1-civicrm_contribution_trigger_as_running_on_live.mysqlNoNoHEAD66 second(s)455 per minute
2-fgb-change-528307-3.mysqlYesNohttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528307/66 second(s)455 per minute.
2-fgb-change-528307-3.mysqlYesNohttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528307/67 second(s)448 per minute
2-fgb-change-528307-3.mysqlYesNohttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528307/67 second(s)448 per minute
3-fgb-with-field-back-528310-2.mysqlYesYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528310/67 second(s)448 per minute.
3-fgb-with-field-back-528310-2.mysqlYesYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528310/67 second(s)448 per minute.
3-fgb-with-field-back-528310-2.mysqlYesYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528310/66 second(s)455 per minute.
3-fgb-with-field-back-528310-2.mysqlYesYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528310/67 second(s)448 per minute
3-fgb-with-field-back-528310-2.mysqlYesYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/528310/66 second(s)455 per minute.
4-not-fgb-withfieldback.mysqlNoYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/535343/66 second(s)455 per minute
4-not-fgb-withfieldback.mysqlNoYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/535343/67 second(s)448 per minute
4-not-fgb-withfieldback.mysqlNoYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/535343/67 second(s)448 per minute
4-not-fgb-withfieldback.mysqlNoYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/535343/67 second(s)448 per minute
4-not-fgb-withfieldback.mysqlNoYeshttps://gerrit.wikimedia.org/r/#/c/wikimedia/fundraising/crm/+/535343/67 second(s)448 per minute.

Change 528310 merged by Eileen:
[wikimedia/fundraising/crm@master] re-add date of largest contribution

https://gerrit.wikimedia.org/r/528310

@Jgreen @Dwisehaupt so the status now is

To get to the point where we are logging all the fields we want to log run

sites/all/modules/wmf_civicrm/scripts/contribution_table_trigger_with_date_of_largest_contribution.mysql

To revert back if there is an issue run

sites/all/modules/wmf_civicrm/scripts/contribution_table_trigger_excluding_date_of_largest_contribution.mysql

Eileenmcnaughton added a subscriber: NNichols.

@NNichols we should be capturing 'date of largest donation' again and I have backfilled missed data - does anyone else need to be informed?

yes please! I was the one using the data primarily but I'll let the team know it's there.

Eileenmcnaughton closed this task as Resolved.Sep 13 2019, 4:58 AM