@Eileenmcnaughton, @RLewis just let me know that you had this discussion about not overwriting data in existing records so that it would not overwrite good data with bad. I'm fine with that as well, just wanted to make sure it wasn't some sort of bug.

I'm still curious about the CID 40129228 example. Even though it is already in civi, is there a way to import that org contact information if the fields are blank? Or is it not possible to specify 'only add data if the field is empty'?

Thanks!

Change 682793 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Keep imported org details on update

https://gerrit.wikimedia.org/r/682793

@RLewis @MDemosWMF I've put up a patch that is a slight variant on what we have discussed - it will ALWAYS add the new values now unless the existing value contains them already - but it will append them - eg.

existing value = ''
imported value = 'Bill'
new value = 'Bill'

existing value = 'Bill'
imported_value = 'Bob'
new value = 'Bill|Bob'

existing value = 'Billy'
imported_value = 'Bill'
new value = 'Billy' (because Bill is in Billy)

existing value = 'Bill'
imported_value = 'Billy'
new value = 'Bill|Billy' (because Billy is not in Bill)

I think this might create slightly messier data - e.g the 'Bill|Billy' scenario - but at least it will all be there for you to review

@Eileenmcnaughton thanks for explaining those scenarios - I think that will work great to capture the new information as well as the old (if they are different), so we're not losing any data. These fields aren't used for mailings, they're mainly internal so I think it's ok if it gets slightly messy to conserve that data. @RLewis let me know if I'm wrong on that.

Does this rule go for the Individual import with the partner field as well? It would be great to have this apply to that field for the Engage Individual import, too. Thank you!

@Eileenmcnaughton and @MDemosWMF thanks both just jumping in here, we do actually use the Org Contact Field for mailings, especially for the Year End Appeal and Annual Report Mailings for our Organizations where we the data in these fields and also for correspondence with a Foundation or other Organization. This was one of the main reasons for adding the fields in. As we often lost this information that was sent in letters with the checks included on the import.

For example, if we have a Foundation who has previously donated $5k and the Grants Officer has changed from one year to the next we will want to keep the new Grants Officer name and override the previous name in this field. What would happen in this case? Would we have both names?

@RLewis thanks for that insight! Seems like having the old data and new data saved in the same field could cause an issue with mailings then. You could either have it overwrite old data like you mentioned or not have the import add new data if there is already information present in that field. In that case the contact wouldn't get updated through imports, so it might be best to overwrite if that's how you plan to get the updated contact info.

@MDemosWMF I think overwriting old data is fine as want the most recent Org Contact detail as agreed that's how we will get the most recent contact information.

@RLewis that sounds good. So @Eileenmcnaughton could we have the new data overwrite the previous data in the org contact fields at each import?

It would be:

existing value = 'Bill'
imported_value = 'Bob'
new value = 'Bob' (no append)

@RLewis Would you would an amount limit on overwriting, for example with $25K+ gifts so they can be reviewed?

@LeanneS and @MDemosWMF yes that would be a good threshold I think. I do tend to do a bit of reviewing for the $5k and over gifts that come in via Engage anyway but the $25k threshold sounds good.

So - we want the behaviour as is for 25k+ & for less than that just overwrite?

@Eileenmcnaughton I think that seems right to me. I tend to do a lot of manual reviewing of donor records and documentation received with checks for these imports to relay back to MG officers and if I notice that we're overwriting a lot of data we really shouldn't be can we adjust the threshold at a later date if need be?

@RLewis this should be working now

@Eileenmcnaughton thanks.