Fields affected in large tables

Slow query 1

UPDATE civicrm_activity SET is_deleted = 0 WHERE is_deleted IS NULL;

We don't install CiviCRM via composer (when we get off Drupal 7 we will) - so we have the entire tarball of CiviCRM on our site (and we just manage it to avoid conflicts with the other packages we have on the Drupal site). CiviCRM runs with Wordpress, Drupal and Joomla which all have different relationships with composer....

@Reedy - what is your feeling on https://github.com/wikimedia/composer-merge-plugin/issues/221 & making the merging of 'replace' configurable per that github issue - which would help us here

Note I tried switching replace to 'self.version' per the docs

-So on 1 - yes the currency code will ALWAYS be there if you replace

I put up a patch for this just now as a good way to test another change https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/801444

@tomasz I'll look more carefully next week but on a general level

@CDenes_WMF yes - we should probably set up a contact in Civi called TestMeister with 1 of each of those donation types - & then people can edit the email address to test & send (& the donation currencies too). All fr-tech staff have their own contacts they use to test donations but a 'TestMeister' contact might be clearer.

@nisrael so - I just tried & it was greyed out & then I selected 'just this page' & it worked & then I went back to all pages & it worked - so there is some glitchiness - possibly because it is selecting in the background

@nisrael yep!

@nisrael - this gets around 100k rows & chooses starting with a-c

@KHaggard - so we could always just try doing some more filters on email - I get 30k-ish if I do 'just' begins with 'a' - we don't have to do only one letter at once - but maybe it makes sense to go through the process on a set this size & build up from there?

@KHaggard @nsirael if we can put them in a group in Civi then I think that is the right starting point to

On the filter question - I've saved the search & added the language filter + filtered out email addresses that don't start with a letter
https://civicrm.wikimedia.org/civicrm/admin/search#/edit/569

@CDenes_WMF the UI for the mailing templates provides previews for each mailing with each of the options (recurring, restarting a donation etc) eg

@nisrael what is the reason to download them all? I thought the idea was to unhold them in CiviCRM & then that would flow into Acoustic - I'm trying to trace this back to what the original goal was as my understanding there was an intention to selectively unblock some domains (like the Comcast one) where we had reason to think they were blocked in error.

@CDenes_WMF - to ALWAYS include the 3 letter code it is {$currency} - you can conditionally include it with an if eg. {if $currency eq 'USD'}{$currency}[/if}

@nisrael is looks like you have the ones who are NOT on hold there

@AnnWF - is it possible you need to update your CiviCRM codebase ? - it looks like the fix was committed in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/civicrm/+/778336/1/vendor/civicrm/composer-compile-plugin/src/TaskRunner.php

This seems to relate - https://github.com/civicrm/composer-compile-plugin/pull/20 - the issue appears to be stricter typing in later version of composer.

I would note that at that time utf8bm4 was a conversion from utf8 which was the old default - however, utf8mb4 is pretty much the MariaDB default now - so if we are on more recent MariaDB now than we were then those lines may no longer be required.

@SHust also - I've added the 'Donor Services Management' role to Adriana

@SHust I checked in with 10827232 & he CAN access the tab - however he could not access the tab for his own record (nor could I) - the error was "Request failed: Recipient not found or is not a member of the list”

• that error means the person is not in the Acoustic db & I think it makes sense that some staff records won't be

It is possible to add {$currency} into the templates to add the 3 letter code - there is some risk that in some cases it will be a duplication but if it's never happening in Spanish then it can be added

@SHust I've put out a patch that covers the new Acoustic screen

Per comments above - we checked in with @KHaggard on our call this morning and she confirmed the main issue is USD not showing up in English - the reason for this is that the server is in 'en_US' locale so USD is being treated as implicit and only non-US currencies are getting locale/currency-relevant codes added.

@KHaggard it is not a big deal from our end to do it - so up to your availability

@MNoorWMF - we will have to do a full database export when we deploy the fix for https://phabricator.wikimedia.org/T305483 - so if anything IS missed they will be imported then

@nisrael sure - but I think you might not need us - you can remove the on hold from contact in bulk from search results (which is what we would do too).

@KHaggard when should we deploy this extra field?

@KHaggard @AnnWF I was looking at the patch Wenjun put up but I think we need to dig a bit deeper

@Ejegg checked the offline2civicrm log table & the 4 in question have not been used since 2018:

I finished uploading @RLewis's group of 1500 contacts and 4 were rejected by Acoustic because the local parts of their emails were (respectively) 'marketing', 'support', 'service' and 'admin'

@RLewis
I have exported your group 1218 to Acoustic

I've put up patches to remove

• Paypal EFT
• Coinbase
• the 4 fr-tech scripts

@AnnWF - I just deployed this patch & ran the upgrade - field seems to work now

@AnnWF - I just deployed this patch

@KHaggard - I just happened to hit one of the handful of contacts with marketing as a prefix - he is in the major gifts list they want to upload to Acoustic in a group to email - I don't know of any other prefixes but I'll try again to upload the list if I can get past that contact

@KHaggard - there are 218 contacts in this category - they all have donations - which I think reflects the way contacts enter our database

I just checked - here are a couple more donors with that start
https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=521945
https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=43560023

Hmm - we did get a significant donation from that not-real-person....

OK - I've compiled the list & some data about them below

This was obviously a good idea years ago but since it hasn't happened in 7 years it seems we won't

aged out

aged out