@SHust I deployed a patch today that should fix this - however you should be aware that you should probably refresh the result set at the end as the match between the deleted contact and their match will have been removed from the screen but there MAY be a new match between their former match and the contact they were merged into created via the merge

This should be fixed up now

@CaitVirtue I just deployed it - so you should be able to see the text now

PR to do the same re-labelling upstream https://github.com/civicrm/civicrm-core/pull/19707

A fix is incorporated in https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/667053

I'm pretty sure this is

I started digging into this just now & created this table with search kit https://civicrm.wikimedia.org/civicrm/search#/display/Engage/Engage_reconcilliation

@CaitVirtue - I dug into this a bit further. I've proposed a patch that make it clearer that a report with no results did actually run (https://github.com/civicrm/civicrm-core/pull/19668)

I've removed 20285807 from our list

I moved this back into triage as getting this done will remove an ongoing pain point and the work involved now is probably less than the ongoing pain

Composer merge plugin is fixed upstream now. Lets

2015 issue that is either fixed or not bothering us

2015 issue that is either fixed or not bothering us

Oh yeah - I think I remember something about that guzzle.

@RLewis so according to that screenshot everything imported except the 20 rows that are in the error file - do they add up to the missing amount?

@RLewis yes sorry - please try again - the first error is on this contact civicrm/contact/view?reset=1&cid=50372691 and is

Just a bunch of test cleanups in review

Merged, works

Working - yay

@AndyRussG - yes - it might not quite get to zero - but the number of contacts left behind by the tests goes from triple digits to double digits with the chain merged - I left the scope as 'all the tests that inherit from the drupalBase test class' & then locked that in with a test that will fail if we start leaving behind test contacts again

@AndyRussG - yes most of the fixes are to reduce the number of failures you get when it is not a brand new install - by improving cleanup

I checked out the first of these & sadly we really haven't had any $5k + gifts this week - so it''s legit empty - although I want to compare with how it would have looked before. I haven't checked the other 2

Now removed

@RLewis please confirm when you have tested so we can close

I put up https://gerrit.wikimedia.org/r/c/integration/config/+/664359 to remove reference to the drupal modules - that needs releng (@hashar ) to merge it - once merged we need to merge https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/664360 and then we will no longer need the drupal repo (which can then be fully removed from the ci config & 'closed' whatever that means)

The flapping is cleanup related - the errors are because we are not setting the collation correctly when we create the db

I just spotted https://github.com/wikimedia/composer-merge-plugin/pull/189 was merged but this ticket hasn't been updated so adding a note (I had been following the ticket & not the gitlab issue which was unfortunate as I didn't realise there was a fix merged)

@MBeat33 ah yes - I recall you've hit that before - it is your browser pre-filling data rather than anything CiviCRM is saving I think from memory

@RLewis yes please

I just tried & saw that message but got results fine.

@RLewis the csv with the error messages should have the detail about the errors - they are per row - but it gave up after hitting 10 rows in a row with errors

Thanks @RLewis I was able to diagnose this & logged an upstream issue here https://lab.civicrm.org/dev/mail/-/issues/89

@RLewis yep - that's how it is set up now

@Dwisehaupt working through this on our CI enviroment we learnt 2 things

@Cstone & I forced the change to the drupal repo through just now in the hope the jenkins problem was just trying to find it's way around applying the changes (it appeared to have issues with the remove & re-add).

@KHaggard sure - we can keep it to track - for some reason I couldn't see the endowment list

@jgleeson no hurry from my end - the other patch in there will remove some failmail - but of a type that is not happening much at the moment anyway

@jgleeson I got a bit confused about the state of vendor when deploying just now - when I ran composer install --no-dev locally I got this diff https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/vendor/+/661537

Cool - I still have to deal with the docker images charset but at least I know prod is done

@Dwisehaupt ALL civicrm db tables should be converted. On new installs the tables are specifically defined with utfmb48 so we are trying to get inline with that.

@Dwisehaupt I think utf8 is better and then change to utfmb4 when we alter the tables. Currently all our tables are utf8 - which we want to change to utf8mb4. However, we need to avoid creating some tables using utf8mb4 accidentally when the rest are on utf8 because the joins won't work between utf8 & utf8mb4 fields

Just noting the report in question is civicrm/report/instance/75 (and 74) based on CRM_Report_Form_Contribute_Summary - I tried just adding the field but that didn't work

Also note the patch I put up is now on staging

@RLewis I've put up a patch to support the following columns in the csv

@RLewis Yes - I just had the Email field right under the non-expanded custom data set so I didn't realise it wasn't expanded!

@Dwisehaupt @Jgreen I just tried to run this on staging civicrm db

Just noting that mysql connection if via phpstorm mysql plugin (not saying we shouldn't do this just that there is an alternative that some people might choose)

I've put up a patch to add Partner in - but I'm less sure about the Org import.

@Dwisehaupt it's possible to see ini variables at this url civicrm/api3#explorer using System & get

I suspect the issue is the sheer volume of contributions caught by 'in the last 3 years' - my guess is that the query itself would be faster if it were forced to use the total_amount index on this particular query - but that would rely on some hard-coding of the query. I will ask Coleman if we can expose index hints for very advanced search use

@jkim_wikimedia these files are imported now - I didn't hold one back for further testing as I'm away for a week now & I thought it would be better to get them all in but I couldn't confirm the new max - it is file size related & 3.3 MB with 33k rows is importing in one go

The mailing id is 69308812

Looks like I just managed a 3,3MB file with 30 k rows - it's all odd