Page MenuHomePhabricator

Civi - Increase limit on email sends to 2000 / enable CiviMail
Closed, ResolvedPublic2 Estimated Story Points

Description

MG Would like to increase the number of emails we can send in CiviMail

https://wikimedia.slack.com/archives/C045WH0QYS2/p1681827781163729

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptApr 24 2023, 2:37 PM

I'm trying to arrange a call between Rosie & Eileen to discuss alternate sending methods.

@RLewis is this still a requirement given convos between you and Eileen?

@AKanji-WMF - i think it would be good to still increase this limit to tide us over as we head into appeal season.

Dwisehaupt set the point value for this task to 2.Jul 24 2023, 8:27 PM
Eileenmcnaughton renamed this task from Civi - Increase limit on email sends to 2000 to Civi - Increase limit on email sends to 2000 / enable CiviMail.Aug 29 2023, 7:52 PM

Testing on staging ...

image.png (648ร—1 px, 125 KB)

Documentation https://docs.civicrm.org/user/en/latest/email/mass-mailings-using-civimail/#choosing-recipients-groups-versus-search-results

I note that

  1. we need to create an Unsubscribe group on live - this is a big of a fake for unsubcscibes.... and unless we want to use it it can just be a group for this purpose with no real management
  2. we need to enable this setting (track_civimail_replies) if we want to track replies (which I think is good)

image.png (177ร—1 px, 95 KB)

  1. it attempts to redirect to the dashboad on save - this doesn't load because of the volume of our data....
  2. I think our domain address might be wrong ? 149 New Montgomery Street - is that the right number?

The loading issue / data volume is an issue / trap

I created an upstream issue https://lab.civicrm.org/dev/core/-/issues/4567 - but I think in the short term we need to rationalise our civicrm_mailing data

We have 22978433 records in civicrm_mailing_job that pre-date 2020 ....

We could

  1. consolidate all our thank you mailing records in civicrm_mailing to 1 & update all the civicrm_mailing_job records
  2. consolidate all our thank you mailing job records in civicrm_mailing_job to be just one job linked to the 1 mailing
  3. update all the civicrm_mailing_event_queue records to link to this single job_id
  4. not create new job ids as we thank - keep re-using

Doing the above should get the queries sane and allow the mailing dashboard to load & allow us to start querying stuff

It is further worth thinking about what we want from the older bounce records etc

My feeling is it might be worth creating activities for bounces less than x years old( or perhaps all 1.3m of them), and the unsubscribes & we should probably delete the delivereds & opens after just a month or 2 (AFAIK we have never looked at them)

+------------------------------------------+-------------+

mailing_tablenumber_rows
civicrm_mailing7075
civicrm_mailing_job50296940
civicrm_mailing_event_queue50094686
civicrm_mailing_event_bounce1305082
civicrm_mailing_event_confirm0
civicrm_mailing_event_delivered50070600
civicrm_mailing_event_forward0
civicrm_mailing_event_opened421457
civicrm_mailing_event_reply402
civicrm_mailing_event_subscribe0
civicrm_mailing_event_trackable_url_open62816
civicrm_mailing_event_unsubscribe9168

QUery for above

SELECT 'civicrm_mailing' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing
UNION
SELECT 'civicrm_mailing_job' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_job
UNION
SELECT 'civicrm_mailing_event_queue' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_queue
UNION
SELECT 'civicrm_mailing_event_bounce' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_bounce
UNION
SELECT 'civicrm_mailing_event_confirm' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_confirm
UNION
SELECT 'civicrm_mailing_event_delivered' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_delivered
UNION
SELECT 'civicrm_mailing_event_forward' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_forward
UNION
SELECT 'civicrm_mailing_event_opened' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_opened
UNION
SELECT 'civicrm_mailing_event_reply' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_reply
UNION
SELECT 'civicrm_mailing_event_subscribe' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_subscribe
UNION
SELECT 'civicrm_mailing_event_trackable_url_open' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_trackable_url_open
UNION
SELECT 'civicrm_mailing_event_unsubscribe' as mailing_table, COUNT(*) as number_rows
FROM civicrm_mailing_event_unsubscribe

I pushed out the setting update today - we had planned to switch to CiviMail rather than increase the setting but then suddenly we were in code freeze & it didn't feel safe anymore

Eileenmcnaughton renamed this task from Civi - Increase limit on email sends to 2000 / enable CiviMail to Civi - Increase limit on email sends to 2000.Mon, Dec 4, 8:24 PM
XenoRyet renamed this task from Civi - Increase limit on email sends to 2000 to Civi - Increase limit on email sends to 2000 / enable CiviMail.Mon, Dec 4, 8:24 PM
XenoRyet closed this task as Resolved.
XenoRyet set Final Story Points to 1.