Just noting that we did some testing today & got some improvement on the speed of the greeting processing - but we need to do more testing to see if it's enough we can process all contact's greetings during donation processing.

Any update you do on a contact will cause their greetings to be populated - at a pinch you can probably do a dummy update from search-kit (we could look into this at fortnightly) on a given group (since I understand these are mostly smaller groups)

Change 824392 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Allow greeting processing on new contacts

https://gerrit.wikimedia.org/r/824392

Change 824392 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Allow greeting processing on new contacts

https://gerrit.wikimedia.org/r/824392

Change 833131 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Use a setting to enable & disable greeting processing on create

https://gerrit.wikimedia.org/r/833131

Change 833131 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Use a setting to enable & disable greeting processing on create

https://gerrit.wikimedia.org/r/833131

Change 833267 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Commit test that was somehow missing from greeting setting commit

https://gerrit.wikimedia.org/r/833267

Note that we have a fix for this going forward, but we have not backfilled existing records. There are 14 million+ contacts in the db with this field empty, so it would take a little while to fill them all.

We would want to have a pretty constantly-running script that just grabs batches of contacts that have this field blank and resaves them (which will calculate the greeting). That will probably end up giving us some database contention.

@Ejegg thanks for looking into this, we don't need records backfilled. We only really need it for donors who fall within the major donor audience (donors who make a gift of $250+ or donor advised donors) which is probably around 30k donors. We have been updating the field when we see it is blank ourselves manually. The main reason we wanted to use the email greeting field over first name is because we have a higher number of Organizations who have been long time donors who we email and it doesn't feel right to address them as Dear Wikimedia Supporter. We also have a number of donors who donate as a couple and customizing this field allows us to address donors as a couple in our emails as well.

For our upcoming campaign we do have a fall back if the field is blank - it's Wikimedia Supporter but Desiree has done a lot of manual updating to ensure we don't have too many blank fields for our email list in this instance.

OK, @RLewis 30k is definitely manageable. We should be able to write a fairly simple script to find donors where email greeting is blank AND (has a contribution > $250 OR has a 'Manages Donor Advised Fund' / 'Is the Donor Advised Fund of' relationship), then resave all of those.

Change 838948 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] WIP script to fill missing email greetings

https://gerrit.wikimedia.org/r/838948

@Ejegg - I think that sounds like a good plan, since Desiree has done a lot of manual clean up for our current major gift email campaign we wouldn't want to over ride the work she's done to ensure this field is filled. Would it make sense to do the script going forward for any new gifts that come in of $250+ and DAF related? Then we should be in better shape for future major gift email campaigns and little manual clean up would be needed on our end.

@RLewis incoming donations are all being coded at the moment -

Regarding updating the others it's possible to bulk update through the UI which I did on this search - & also one which caught relationships

https://civicrm.wikimedia.org/civicrm/admin/search#/create/Contact?params=%7B%22version%22:4,%22select%22:%5B%22id%22,%22display_name%22%5D,%22orderBy%22:%7B%7D,%22where%22:%5B%5B%22is_deceased%22,%22%3D%22,false%5D,%5B%22email_greeting_display%22,%22IS%20EMPTY%22%5D,%5B%22email_greeting_id:name%22,%22IS%20EMPTY%22%5D,%5B%22contact_type:name%22,%22%3D%22,%22Individual%22%5D%5D,%22groupBy%22:%5B%5D,%22join%22:%5B%5B%22Contribution%20AS%20Contact_Contribution_contact_id_01%22,%22INNER%22,%5B%22id%22,%22%3D%22,%22Contact_Contribution_contact_id_01.contact_id%22%5D,%5B%22Contact_Contribution_contact_id_01.total_amount%22,%22%3E%22,%22%5C%22249%5C%22%22%5D%5D%5D,%22having%22:%5B%5D%7D

What I found (which is what @Ejegg hit) - some have a NULL email_greeting_id - so I did the update like this

I'm not sure if I captured all the contacts you are looking at - but any that can be found through search kit could be updated with the same process maybe? It definitely copes with searches that return 5k rows but 800k rows not so much :-)

@Eileenmcnaughton amazing, thanks for the update Eileen. This will save a lot of manual clean up for future major gift mailings.

I'm moving this to done - but I'm gonna create a follow on to back fill greetings more generally