Hi @Eileenmcnaughton - putting this high chaos item from @SHust on your radar - TY

wow, the DB error is a new one for me: The table 'XXX' is full
@Dwisehaupt or @Jgreen is this related to disk space?

INSERT IGNORE INTO civicrm_tmp_e_logsummary_84af1a7e1dec9b95b5c998a045d4fcce SELECT activity_id,  IF (entity_log_civireport.log_action = 'Insert' AND extra_table.activity_type_id = 84 , GROUP_CONCAT(entity_log_civireport.contact_id), 1) , entity_log_civireport.log_action as log_civicrm_entity_log_action, 'log_civicrm_activity_contact' as log_civicrm_entity_log_type, entity_log_civireport.log_user_id as log_civicrm_entity_log_user_id, entity_log_civireport.log_date as log_civicrm_entity_log_date, modified_contact_civireport.display_name as log_civicrm_entity_altered_contact, modified_contact_civireport.id as log_civicrm_entity_altered_contact_id, entity_log_civireport.log_conn_id as log_civicrm_entity_log_conn_id, modified_contact_civireport.is_deleted as log_civicrm_entity_is_deleted, altered_by_contact_civireport.display_name as altered_by_contact_display_name  
         FROM `civicrm`.log_civicrm_activity_contact entity_log_civireport
         INNER JOIN civicrm_contact modified_contact_civireport
                 ON (entity_log_civireport.contact_id = modified_contact_civireport.id )
         INNER JOIN `civicrm`.log_civicrm_activity extra_table ON extra_table.id = entity_log_civireport.activity_id
         LEFT  JOIN civicrm_contact altered_by_contact_civireport
                 ON (entity_log_civireport.log_user_id = altered_by_contact_civireport.id) WHERE ( modified_contact_civireport.id = 51560991 ) AND (entity_log_civireport.log_action != 'Initialization') GROUP BY entity_log_civireport.log_conn_id, entity_log_civireport.log_user_id, EXTRACT(DAY_MICROSECOND FROM entity_log_civireport.log_date), entity_log_civireport.id  ORDER BY entity_log_civireport.log_date DESC  [nativecode=1114 ** The table 'civicrm_tmp_e_logsummary_84af1a7e1dec9b95b5c998a045d4fcce' is full

Here's the exact timing:

Jun 10 19:33:30 frdb1005 mariadbd[1336]: 2025-06-10 19:33:30 1006349 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_84af1a7e1dec9b95b5c998a045d4fcce' is full

More:

Jun  4 14:48:12 frdb1005 mariadbd[1336]: 2025-06-04 14:48:12 573461 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_fce2ff693c6a077c755850d3e88067a7' is full
Jun  4 14:50:03 frdb1005 mariadbd[1336]: 2025-06-04 14:50:03 573507 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_5937032ab3fbedec79f00b1245103117' is full
Jun  4 14:53:51 frdb1005 mariadbd[1336]: 2025-06-04 14:53:51 573668 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_57d04af4967304e4d77ea086056ed559' is full
Jun  4 15:05:21 frdb1005 mariadbd[1336]: 2025-06-04 15:05:21 574075 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_b1f5837ecc0d1830139d3500d8b4d8a5' is full
Jun  9 15:02:29 frdb1005 mariadbd[1336]: 2025-06-09 15:02:29 886303 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_0c92d2c08572fa07daafb43ebe1d43e5' is full
Jun 10 19:33:30 frdb1005 mariadbd[1336]: 2025-06-10 19:33:30 1006349 [ERROR] mariadbd: The table 'civicrm_tmp_e_logsummary_84af1a7e1dec9b95b5c998a045d4fcce' is full

possibly the crazy amount of activities they are linked to....

Looks like almost 73k activities for that DR person. But I can select it and dump it to screen without any issue.

Perhaps we can add a setting to limit the UI view of those change logs to the last N activities.

I can make the page load on staging by adding a date limit to the where function in CRM/Logging/ReportSummary.php, but I have to restrict it to the last 6 months - one year is still getting the 'table full' error.

But... we probably don't want a limit that short for most contacts' change logs. We could use a numeric limit instead, but that would end up looking strange, because it's doing selects from several tables to combine into this report, and limiting to e.g. 100 activities + 100 contact changes + 100 email changes means each set of changes would go back a different length of time into the past.

If instead I change it to make the temp table durable - on disk instead of in memory - it loads the report pretty quickly, with all 72k+ records paginated. But I'm not sure when that temp table would be cleaned up. @Eileenmcnaughton do you know?

@Dwisehaupt or @Jgreen
Would it be possible to adjust some of the parameters mentioned here?
https://dba.stackexchange.com/questions/198569/the-table-is-full-mariadb
It looks like in our case tmp_table_size and max_heap_table_size are pretty small - just 16M

I'm not sure if it will make a huge difference but I did bump up those two values. Give it another shot and see if there is a change in behavior.

Sadly, still timing out. @Eileenmcnaughton if we wanted to change that temp table to be durable rather than in memory, do you know the best way to later clean it up? Is there some automatic tracking and cleanup of non-memory temp tables?

@Ejegg I don't know if we should be trying to solve this problem - that change log is not really designed for users with this volume of activity - @SHust what are you wanting to get out of viewing the change log for your staff - maybe we can skin the cat (meow) a different way

@Eileenmcnaughton, thanks for clarifying! What I was hoping to get from the change log is a straightforward way to review the team’s activity by agent/CID when I need to troubleshoot or verify case handling. If that tool isn’t meant for high-volume use, I’m open to exploring other options, whether that’s a different report, dashboard, or process. I'm happy to hear suggestions on what might provide similar visibility. Thank you.

@SHust Maybe I can jump in here. I agree with @Eileenmcnaughton that the change log tends to be painfully slow when dealing with this volume of records. But your use case does seem like something we might be able to address with a SearchKit / FormBuilder. Would you like to provide some details here about what you're looking to see specifically for each agent or should we set up a call to discuss?

@Lars, a call sounds great, thanks for the offer!

Just adding some notes here that the specific ask is for a way to track when someone makes a change to the Email Amender (which is currently not available via Change Log) and to combine that with data showing changes to contacts since that change was made (to ensure we are handling email address correction for existing contacts). @SHust please correct me if I'm off base here.

Also, how are the existing contact email changes being made? How are these being done in bulk?

Hi, @Lars, your notes are correct!
Regarding the current process, when possible, a list of contacts in need of updating is pulled and then placed into a group for bulk updating.

@SHust Unfortunately, pulling in the details of changes to the Email Amender rules will be a bigger project than it sounds like it should be as this information isn't available in SearchKit. Changes to emails are also not available in SearchKit. All we have available currently is that a particular contact modified another contact, but we can't access the specific changes. I can make you a search form to show a list of contacts modified by a specific contact, if that is helpful.

In the longer term, we'd like to add all of the log tables to SearchKit, but it will be a bit of a project, so not something we can do in the short term.

For the current process, I just wanted to make sure there is a non-painful method to do this in bulk. Let me know if not.

In discussing with @SBurnett-WMF and looking at our roadmap, we propose to review this again mid Q3. Let us know if this poses issues.