Page MenuHomePhabricator

slow anonymous save
Closed, ResolvedPublic2 Estimated Story Points

Description

Hi Eileen,

Please see Engage's note below re: slowness. Could you look into this for them?

Thanks!
Leanne

  • Forwarded message ---------

From: Chris C
Date: Mon, Jul 16, 2018 at 10:05 AM
Subject: 'Anonymous Anonymous' entries in Civi
To: Leanne Schreibstein

Hi again Leanne -

When keying entries for the ‘Anonymous Anonymous’ record in Civi, we are seeing longer-than-usual wait times for the entries to save. It sometimes takes a full minute for the entries to save. There has always been a delay for these types of entries - maybe 10 seconds or so - but over the past few days, they seems to be taking longer.

Can you take a look, and maybe have tech look into?

Thanks,

Chris

Event Timeline

Query on view contact for count

   SELECT   COUNT(DISTINCT(tbl.activity_id)) as count
     FROM   (  
            SELECT civicrm_activity.id as activity_id
            
            from civicrm_activity
            left join civicrm_option_value on
                civicrm_activity.activity_type_id = civicrm_option_value.value
            left join civicrm_option_group on
                civicrm_option_group.id = civicrm_option_value.option_group_id
            
INNER JOIN civicrm_activity_contact ac ON ac.activity_id = civicrm_activity.id
INNER JOIN civicrm_contact contact ON ac.contact_id = contact.id

            where
                     ac.contact_id = 72 
                AND civicrm_option_group.name = 'activity_type' AND civicrm_activity.is_deleted = 0 AND civicrm_activity.is_current_revision =  1 AND civicrm_activity.is_test= 0 AND ( civicrm_option_value.component_id IS NULL OR civicrm_option_value.component_id IN ( 1,2,4,8 ) )
           ) as tbl
LEFT JOIN   civicrm_case_activity ON ( civicrm_case_activity.activity_id = tbl.activity_id )
    WHERE   civicrm_case_activity.id IS NULL

@Eileenmcnaughton I just received a note from Chris that the anonymous record isn't working at all for them now. It shows as a blank page for them. Could this have to do with the deploy?

@LeanneS I figured out what is happening - the php timeout on live is only 30 seconds - it's being exceeded when the record is loaded (due to too many activities). The mysql time is much better but the php time is as issue on live (only)

@LeanneS @cwdent has increased the time out at the php level so the page loads - I do have another path to pursue to speed it up more - discussed on https://github.com/civicrm/civicrm-core/pull/12557

Change 450177 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm@master] Revert "Activity tab performance - switch from deprecatedGetActivitiesCount to getActivitiesCount"

https://gerrit.wikimedia.org/r/450177

Change 450177 abandoned by Ejegg:
Revert "Activity tab performance - switch from deprecatedGetActivitiesCount to getActivitiesCount"

Reason:
Went for the real fix instead, seems to work great!

https://gerrit.wikimedia.org/r/450177

@Eileenmcnaughton @cwdent Thanks for your help with this fix. The latest from Chris is that the change sped up the page load to get into the Anonymous record, however, it still is slow to save entries.

@LeanneS I just tried to replicate this on staging & wasn't sure that I was seeing as described. What does Engage actually do when they 'key entries' - I tried going to the anonymous contact & doing 'new contribution'

So I did some stats on this after they sent me a screencast to replicate steps.

I think there are 3 contributors to slowness - 2 affect anonymous user in particular

  1. slow contribution save due to summary stats (wmf_donor) calculation
  2. slow contribution list rendering due to presentation layer summary stats calculation
  3. multiple unnecessary small queries

Slow contribution save due to summary stats (wmf_donor) calculation

It takes 11 seconds to save a contribution for the anonymous donor. This really is the length of time the calculations take to run on a contact with 18000+ contributions. For most contacts it is fast. I am thinking that it might be possible to do an early return specifically for this one contact id (72). Potentially also for the endowment financial type on our main donor update.

Slow contribution list rendering due to presentation layer summary stats calculation
I've raised this upstream to address 1/2 of the relevant queries
https://github.com/civicrm/civicrm-core/pull/12810

multiple unnecessary small queries
I think we can adopt an approach of having a layer of Php caching backed by Redis. Php caching is faster than Redis but if there is a cache miss then getting from Redis is notably faster than from mysql. I need to discuss this approach a bit more with Tim to agree how it will look

Analysis of queries (until I got bored)

Querycounttimetakentotal
insert contribution11111
render summary stats26.513
get entered cont1<1
get campaigns3<11
get customfields1<1
get menu3<1
get domain13<1
get components8<1
get product2<1
get financial type3<1
get location type5<1

I got bored before doing all of them - however, from my own UI testing getting rid of those small calls & using a combo of php caching & Redis makes a noticeable difference to user experience - there are as many as 40 calls in the list above that could be eliminated.

Query list

Count: 1  Time=11.26s (11s)  Lock=0.00s (0s)  Rows=0.0 (0),

/* https://oursite/user/N */ INSERT INTO civicrm_contribution (contact_id , financial_type_id , payment_instrument_id , receive_date , non_deductible_amount , total_amount , fee_amount , net_amount , trxn_id , currency , cancel_date , cancel_reason , receipt_date , thankyou_date , source , is_pay_later , contribution_status_id , check_number ) VALUES ( N ,  N ,  N ,  N ,  N ,  N ,  N ,  N , 'S' , 'S' ,  NULL ,  NULL ,  N ,  NULL , 'S' ,  N ,  N ,  NULL )

Count: 1  Time=6.34s (6s)  Lock=0.00s (0s)  Rows=1.0 (1),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT COUNT( conts.total_amount ) as total_count,
SUM(   conts.total_amount ) as total_amount,
AVG(   conts.total_amount ) as total_avg,
conts.currency              as currency, SUBSTRING_INDEX(GROUP_CONCAT(conts.total_amount
ORDER BY conts.civicrm_contribution_total_amount_count DESC SEPARATOR 'S'), 'S', N) as amount,
MAX(conts.civicrm_contribution_total_amount_count) as civicrm_contribution_total_amount_count
FROM (SELECT civicrm_contribution.total_amount, COUNT(civicrm_contribution.total_amount) as civicrm_contribution_total_amount_count,
civicrm_contribution.currency  FROM civicrm_contact contact_a LEFT JOIN civicrm_contribution ON civicrm_contribution.contact_id = contact_a.id  WHERE  ( contact_a.id = 'S' )  AND (contact_a.is_deleted = N) AND (contact_a.is_deleted = N) AND civicrm_contribution.contribution_status_id = N
GROUP BY currency, civicrm_contribution.total_amount ORDER BY civicrm_contribution_total_amount_count DESC) as conts
GROUP BY currency

Count: 1  Time=6.30s (6s)  Lock=0.00s (0s)  Rows=1.0 (1),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT COUNT( conts.total_amount ) as total_count,
SUM(   conts.total_amount ) as total_amount,
AVG(   conts.total_amount ) as total_avg,
conts.currency              as currency FROM (
SELECT civicrm_contribution.total_amount, COUNT(civicrm_contribution.total_amount) as civicrm_contribution_total_amount_count,
civicrm_contribution.currency  FROM civicrm_contact contact_a LEFT JOIN civicrm_contribution ON civicrm_contribution.contact_id = contact_a.id  WHERE  ( contact_a.id = 'S' )  AND (contact_a.is_deleted = N) AND (contact_a.is_deleted = N) AND civicrm_contribution.contribution_status_id = N  GROUP BY civicrm_contribution.id
) as conts
GROUP BY currency

Count: 1  Time=0.12s (0s)  Lock=0.00s (0s)  Rows=50.0 (50),
# Full_scan: No  Full_join: Yes  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: Yes
SET timestamp=N;
/* https://oursite/user/N */ SELECT  civicrm_contribution.id, contact_a.id as contact_id, contact_a.contact_type as `contact_type`, contact_a.contact_sub_type as `contact_sub_type`, contact_a.sort_name as `sort_name`, civicrm_contribution.id as contribution_id, civicrm_contribution.currency as `currency`, civicrm_contribution.receive_date as `receive_date`, civicrm_contribution.total_amount as `total_amount`, civicrm_contribution.cancel_date as `cancel_date`, civicrm_contribution.thankyou_date as `thankyou_date`, civicrm_contribution.source as `contribution_source`, civicrm_contribution.contribution_recur_id as `contribution_recur_id`, civicrm_contribution.is_test as `is_test`, civicrm_contribution.is_pay_later as `is_pay_later`, civicrm_contribution.contribution_status_id as `contribution_status_id`, civicrm_contribution.campaign_id as `contribution_campaign_id`, civicrm_financial_type.id as financial_type_id, civicrm_financial_type.name as financial_type, civicrm_product.id as product_id, civicrm_product.name as `product_name`, civicrm_product.id as `contribution_product_id`  FROM civicrm_contact contact_a LEFT JOIN civicrm_contribution ON civicrm_contribution.contact_id = contact_a.id  INNER JOIN civicrm_financial_type ON civicrm_contribution.financial_type_id = civicrm_financial_type.id  LEFT  JOIN civicrm_contribution_product ON civicrm_contribution_product.contribution_id = civicrm_contribution.id LEFT  JOIN civicrm_product ON civicrm_contribution_product.product_id =civicrm_product.id  WHERE  ( contact_a.id = 'S' )  AND (contact_a.is_deleted = N)   GROUP BY civicrm_contribution.id  ORDER BY `receive_date` desc, `contact_a`.`id`  LIMIT N, N

Count: 1  Time=0.04s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT COUNT( conts.total_amount ) as cancel_count,
SUM(   conts.total_amount ) as cancel_amount,
AVG(   conts.total_amount ) as cancel_avg,
conts.currency              as currency FROM (
SELECT civicrm_contribution.total_amount, civicrm_contribution.currency  FROM civicrm_contact contact_a LEFT JOIN civicrm_contribution ON civicrm_contribution.contact_id = contact_a.id  WHERE  ( contact_a.id = 'S' )  AND (contact_a.is_deleted = N) AND (contact_a.is_deleted = N) AND civicrm_contribution.cancel_date IS NOT NULL
GROUP BY civicrm_contribution.id
) as conts
GROUP BY currency

Count: 2  Time=0.01s (0s)  Lock=0.00s (0s)  Rows=2159.0 (4318),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT  camp.id,
camp.title,
camp.start_date,
camp.end_date
FROM  civicrm_campaign camp
WHERE  ( camp.title IS NOT NULL ) AND ( camp.is_active = N )
Order By  camp.title

Count: 1  Time=0.01s (0s)  Lock=0.00s (0s)  Rows=2159.0 (2159),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT  camp.id,
camp.title,
camp.start_date,
camp.end_date
FROM  civicrm_campaign camp
WHERE  ( N )
Order By  camp.title

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=37.0 (37),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT civicrm_custom_field.id as civicrm_custom_field_id, civicrm_custom_field.name as civicrm_custom_field_name, civicrm_custom_field.label as civicrm_custom_field_label, civicrm_custom_field.column_name as civicrm_custom_field_column_name, civicrm_custom_field.data_type as civicrm_custom_field_data_type, civicrm_custom_field.html_type as civicrm_custom_field_html_type, civicrm_custom_field.default_value as civicrm_custom_field_default_value, civicrm_custom_field.attributes as civicrm_custom_field_attributes, civicrm_custom_field.is_required as civicrm_custom_field_is_required, civicrm_custom_field.is_view as civicrm_custom_field_is_view, civicrm_custom_field.help_pre as civicrm_custom_field_help_pre, civicrm_custom_field.help_post as civicrm_custom_field_help_post, civicrm_custom_field.options_per_line as civicrm_custom_field_options_per_line, civicrm_custom_field.start_date_years as civicrm_custom_field_start_date_years, civicrm_custom_field.end_date_years as civicrm_custom_field_end_date_years, civicrm_custom_field.date_format as civicrm_custom_field_date_format, civicrm_custom_field.time_format as civicrm_custom_field_time_format, civicrm_custom_field.option_group_id as civicrm_custom_field_option_group_id, civicrm_custom_field.in_selector as civicrm_custom_field_in_selector, civicrm_custom_group.id as civicrm_custom_group_id, civicrm_custom_group.name as civicrm_custom_group_name, civicrm_custom_group.table_name as civicrm_custom_group_table_name, civicrm_custom_group.title as civicrm_custom_group_title, civicrm_custom_group.help_pre as civicrm_custom_group_help_pre, civicrm_custom_group.help_post as civicrm_custom_group_help_post, civicrm_custom_group.collapse_display as civicrm_custom_group_collapse_display, civicrm_custom_group.style as civicrm_custom_group_style, civicrm_custom_group.is_multiple as civicrm_custom_group_is_multiple, civicrm_custom_group.extends as civicrm_custom_group_extends, civicrm_custom_group.extends_entity_column_id as civicrm_custom_group_extends_entity_column_id, civicrm_custom_group.extends_entity_column_value as civicrm_custom_group_extends_entity_column_value, civicrm_custom_group.max_multiple as civicrm_custom_group_max_multiple, civicrm_custom_group.is_public as civicrm_custom_group_is_public
FROM     civicrm_custom_group
LEFT JOIN civicrm_custom_field ON (civicrm_custom_field.custom_group_id = civicrm_custom_group.id)
WHERE civicrm_custom_group.is_active = N
AND civicrm_custom_field.is_active = N
AND civicrm_custom_group.extends IN ('S')
AND civicrm_custom_group.extends_entity_column_value IS NULL AND civicrm_custom_group.style = 'S' AND  ( N )
ORDER BY civicrm_custom_group.weight,
civicrm_custom_group.title,
civicrm_custom_field.weight,
civicrm_custom_field.label

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=42.0 (42),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT civicrm_custom_field.id as civicrm_custom_field_id, civicrm_custom_field.name as civicrm_custom_field_name, civicrm_custom_field.label as civicrm_custom_field_label, civicrm_custom_field.column_name as civicrm_custom_field_column_name, civicrm_custom_field.data_type as civicrm_custom_field_data_type, civicrm_custom_field.html_type as civicrm_custom_field_html_type, civicrm_custom_field.default_value as civicrm_custom_field_default_value, civicrm_custom_field.attributes as civicrm_custom_field_attributes, civicrm_custom_field.is_required as civicrm_custom_field_is_required, civicrm_custom_field.is_view as civicrm_custom_field_is_view, civicrm_custom_field.help_pre as civicrm_custom_field_help_pre, civicrm_custom_field.help_post as civicrm_custom_field_help_post, civicrm_custom_field.options_per_line as civicrm_custom_field_options_per_line, civicrm_custom_field.start_date_years as civicrm_custom_field_start_date_years, civicrm_custom_field.end_date_years as civicrm_custom_field_end_date_years, civicrm_custom_field.date_format as civicrm_custom_field_date_format, civicrm_custom_field.time_format as civicrm_custom_field_time_format, civicrm_custom_field.option_group_id as civicrm_custom_field_option_group_id, civicrm_custom_field.in_selector as civicrm_custom_field_in_selector, civicrm_custom_group.id as civicrm_custom_group_id, civicrm_custom_group.name as civicrm_custom_group_name, civicrm_custom_group.table_name as civicrm_custom_group_table_name, civicrm_custom_group.title as civicrm_custom_group_title, civicrm_custom_group.help_pre as civicrm_custom_group_help_pre, civicrm_custom_group.help_post as civicrm_custom_group_help_post, civicrm_custom_group.collapse_display as civicrm_custom_group_collapse_display, civicrm_custom_group.style as civicrm_custom_group_style, civicrm_custom_group.is_multiple as civicrm_custom_group_is_multiple, civicrm_custom_group.extends as civicrm_custom_group_extends, civicrm_custom_group.extends_entity_column_id as civicrm_custom_group_extends_entity_column_id, civicrm_custom_group.extends_entity_column_value as civicrm_custom_group_extends_entity_column_value, civicrm_custom_group.max_multiple as civicrm_custom_group_max_multiple, civicrm_custom_group.is_public as civicrm_custom_group_is_public
FROM     civicrm_custom_group
LEFT JOIN civicrm_custom_field ON (civicrm_custom_field.custom_group_id = civicrm_custom_group.id)
WHERE civicrm_custom_group.is_active = N
AND civicrm_custom_field.is_active = N
AND civicrm_custom_group.extends IN ('S')
AND (((civicrm_custom_group.extends_entity_column_value LIKE "S"))  OR civicrm_custom_group.extends_entity_column_value IS NULL )
AND civicrm_custom_group.style = 'S' AND  ( N )
ORDER BY civicrm_custom_group.weight,
civicrm_custom_group.title,
civicrm_custom_field.weight,
civicrm_custom_field.label

Count: 3  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (3),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: Yes
SET timestamp=N;
/* https://oursite/user/N */
(
SELECT *
FROM     civicrm_menu
WHERE    path in ( 'S', 'S', 'S', 'S' )
AND domain_id = N
ORDER BY length(path) DESC
LIMIT    N
)
UNION (
SELECT *
FROM   civicrm_menu
WHERE  path IN ( 'S' )
AND domain_id = N
)

Count: 5  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (5),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: Yes
SET timestamp=N;
/* https://oursite/user/N */
(
SELECT *
FROM     civicrm_menu
WHERE    path in ( 'S', 'S', 'S' )
AND domain_id = N
ORDER BY length(path) DESC
LIMIT    N
)
UNION (
SELECT *
FROM   civicrm_menu
WHERE  path IN ( 'S' )
AND domain_id = N
)

Count: 2  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (2),
# Full_scan: Yes  Full_join: No  Tmp_table: Yes  Tmp_table_on_disk: Yes
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: Yes
SET timestamp=N;
/* https://oursite/user/N */
(
SELECT *
FROM     civicrm_menu
WHERE    path in ( 'S', 'S' )
AND domain_id = N
ORDER BY length(path) DESC
LIMIT    N
)
UNION (
SELECT *
FROM   civicrm_menu
WHERE  path IN ( 'S' )
AND domain_id = N
)

Count: 10  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=149.0 (1490),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
use dev_civicrm;
SET timestamp=N;
/* https://oursite/user/N */ SELECT id, name, value, domain_id, contact_id, is_domain, component_id, created_date, created_id
FROM civicrm_setting
WHERE (domain_id = N) AND (is_domain = N)

Count: 2  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT fa.tax_rate, efa.entity_id
FROM civicrm_entity_financial_account efa
INNER JOIN civicrm_financial_account fa ON fa.id = efa.financial_account_id
WHERE efa.entity_table = 'S'
AND efa.account_relationship IN (N )
AND fa.is_active = N

Count: 2  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
# Full_scan: Yes  Full_join: Yes  Tmp_table: Yes  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
SELECT
DISTINCT ( price_set_id ) as id, s.title
FROM
civicrm_price_set s
INNER JOIN civicrm_price_field f ON f.price_set_id = s.id
INNER JOIN civicrm_price_field_value v ON v.price_field_id = f.id
WHERE
is_quick_config = N  AND s.is_active = N  AND s.extends LIKE 'S'  AND s.financial_type_id IN (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N) AND v.financial_type_id IN (N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N,N)  GROUP BY s.id

Count: 3  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=186.0 (558),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT name AS id, symbol AS label FROM civicrm_currency ORDER BY symbol


Count: 3  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=10.0 (30),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: Yes  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT id AS id, display_name AS label FROM civicrm_location_type WHERE is_active = N ORDER BY display_name

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (1),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT a.id as `id`, a.financial_account_id as `financial_account_id`
FROM civicrm_entity_financial_account a
WHERE (a.entity_table = "S") AND (a.entity_id = "S")
LIMIT N
OFFSET N

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=6.0 (6),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT a.id as `id`, a.entity_table as `entity_table`, a.entity_id as `entity_id`, a.account_relationship as `account_relationship`, a.financial_account_id as `financial_account_id`
FROM civicrm_entity_financial_account a
WHERE (a.entity_id = "S") AND (a.entity_table = "S")
LIMIT N
OFFSET N


Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=186.0 (186),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT name FROM civicrm_currency

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
/* https://oursite/user/N */
UPDATE civicrm_group g
SET    cache_date = NULL,
refresh_date = NULL
WHERE  g.cache_date <= 'S'

Count: 2  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=122.0 (244),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT id AS id, name AS label FROM civicrm_option_group ORDER BY name

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (1),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT  id
FROM civicrm_uf_group
WHERE (  civicrm_uf_group.name  = 'S' )

Count: 5  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=10.0 (50),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT id AS id, name AS label FROM civicrm_location_type ORDER BY name

Count: 3  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (3),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT *
FROM civicrm_financial_type
WHERE (  civicrm_financial_type.name  = 'S' )

Count: 2  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT *
FROM civicrm_product
WHERE (  civicrm_product.is_active = N )

Count: 8  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=9.0 (72),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT *
FROM civicrm_component

Count: 13  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=1.0 (13),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */ SELECT *
FROM civicrm_domain

Count: 1  Time=0.00s (0s)  Lock=0.00s (0s)  Rows=0.0 (0),
# Full_scan: Yes  Full_join: No  Tmp_table: No  Tmp_table_on_disk: No
# Filesort: No  Filesort_on_disk: No  Merge_passes: N  Priority_queue: No
SET timestamp=N;
/* https://oursite/user/N */
DELETE gc
FROM civicrm_group_contact_cache gc
INNER JOIN civicrm_group g ON g.id = gc.group_id
WHERE g.cache_date <= 'S'

Change 459887 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Exclude anonymous contact from triggers.

https://gerrit.wikimedia.org/r/459887

Change 459887 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Exclude anonymous contact from triggers.

https://gerrit.wikimedia.org/r/459887

Change 460187 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Updated triggers.

https://gerrit.wikimedia.org/r/460187

Change 460187 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Updated triggers.

https://gerrit.wikimedia.org/r/460187

@LeanneS so - we deployed a fix the other day - but we ALSO deployed a bug which would have more than cancelled the fix out. Hopefully there is some improvement now!

Engage just reported that the anonymous record isn't working for them today. From one of their staff: "I am unable to record donations in anonymous this afternoon. When I save the donation it just spins forever. I refreshed after about 5 minutes. This is happening consistently today."

@LeanneS where are they at on this? I should note that there are 3 known points of slowness

  1. calculating donor totals for contacts with large numbers of donations - we actively excluded Anonymous from this as part of this ticket
  2. loading the activity tab when there are records with large numbers of related contacts
  3. loading the contribution tab when there are large numbers of contributions

We also get temporary slowness when the server is under load.

I'm trying to figure out what I should be focussing on here