I'm trying to arrange a call between Rosie & Eileen to discuss alternate sending methods.

@RLewis is this still a requirement given convos between you and Eileen?

@AKanji-WMF - i think it would be good to still increase this limit to tide us over as we head into appeal season.

Testing on staging ...

Documentation https://docs.civicrm.org/user/en/latest/email/mass-mailings-using-civimail/#choosing-recipients-groups-versus-search-results

I note that

1. we need to create an Unsubscribe group on live - this is a big of a fake for unsubcscibes.... and unless we want to use it it can just be a group for this purpose with no real management
2. we need to enable this setting (track_civimail_replies) if we want to track replies (which I think is good)

3. it attempts to redirect to the dashboad on save - this doesn't load because of the volume of our data....
4. I think our domain address might be wrong ? 149 New Montgomery Street - is that the right number?

The loading issue / data volume is an issue / trap

I created an upstream issue https://lab.civicrm.org/dev/core/-/issues/4567 - but I think in the short term we need to rationalise our civicrm_mailing data

We have 22978433 records in civicrm_mailing_job that pre-date 2020 ....

We could

1. consolidate all our thank you mailing records in civicrm_mailing to 1 & update all the civicrm_mailing_job records
2. consolidate all our thank you mailing job records in civicrm_mailing_job to be just one job linked to the 1 mailing
3. update all the civicrm_mailing_event_queue records to link to this single job_id
4. not create new job ids as we thank - keep re-using

Doing the above should get the queries sane and allow the mailing dashboard to load & allow us to start querying stuff

It is further worth thinking about what we want from the older bounce records etc

My feeling is it might be worth creating activities for bounces less than x years old( or perhaps all 1.3m of them), and the unsubscribes & we should probably delete the delivereds & opens after just a month or 2 (AFAIK we have never looked at them)

+------------------------------------------+-------------+

QUery for above

SELECT 'civicrm_mailing' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing
UNION
SELECT 'civicrm_mailing_job' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_job
UNION
SELECT 'civicrm_mailing_event_queue' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_queue
UNION
SELECT 'civicrm_mailing_event_bounce' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_bounce
UNION
SELECT 'civicrm_mailing_event_confirm' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_confirm
UNION
SELECT 'civicrm_mailing_event_delivered' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_delivered
UNION
SELECT 'civicrm_mailing_event_forward' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_forward
UNION
SELECT 'civicrm_mailing_event_opened' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_opened
UNION
SELECT 'civicrm_mailing_event_reply' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_reply
UNION
SELECT 'civicrm_mailing_event_subscribe' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_subscribe
UNION
SELECT 'civicrm_mailing_event_trackable_url_open' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_trackable_url_open
UNION
SELECT 'civicrm_mailing_event_unsubscribe' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_unsubscribe

I pushed out the setting update today - we had planned to switch to CiviMail rather than increase the setting but then suddenly we were in code freeze & it didn't feel safe anymore