Page MenuHomePhabricator

Deduper and legacy merging overwriting issue
Closed, ResolvedPublic

Description

Newer CID 5273764 opt-in; Older CID 952021 opt-out. After the merge, Civi saved the newer CID into the oldest as an opt-out instead of adjusting it to reflect what was attached to the newer CID. At the moment this change isn’t reflected at Acoustic, however, tomorrow it will be.

CIDs examples not yet merged:

  • Newer CID 60089976 opt-in Older CID 143328 = After the merge Civi will save the newer CID as opt-out instead of keeping the option from the recent CID
  • Newer Contact ID: 60084551 opt-in Older CID 134613 = After the merge Civi will save the newer CID as opt-out instead of keeping the option from the recent CID

It is not scalable to manually remove the opt-out tags from Civi after each merge that saves the incorrect option, and the deduper interface doesn’t show us when an email merged was incorrectly saved as opt-out. Can the overwrite or a mechanism be implemented to prioritize the option from the latest CID over the older one?

Screenshot 2023-08-29 at 2.16.00 PM.png (474×1 px, 78 KB)

Event Timeline

Sandra has flagged this is relatively urgent - moving to Sprint+1; cc @Eileenmcnaughton

Change 967066 had a related patch set uploaded (by Wfan; author: Wfan):

[wikimedia/fundraising/crm@master] Use latest cid is_opt_id option as default

https://gerrit.wikimedia.org/r/967066

@Eileenmcnaughton Here is the bug fix patch for showing checkbox for bool upstream (https://github.com/civicrm/civicrm-core/pull/28069)

@SHust we probably need to clarify this a bit

There are 2 scenarios where you dedupe

  1. Where the contacts have the same email address. These are already opted out in Acoustic if ANY of the contacts with that email address are opted out. Changing the opt out flag will not opt them back in again & and will just reflect their status incorrectly in CiviCRM. The setting of the opt out based on ANY rather than the most recent is by design (the instructions originally came from the online email team - although I think the staff have left now) - I updated the docs a bit to be clearer
  1. Where the contacts have different email addresses

If 2 is the problem you are trying to solve then that makes sense & we can look to address that in this phab

If 1 is the problem you are hoping to solve then we need to open up a much bigger discussion with stakeholders. There are both technical issues (Acoustic exports current can't remove the opt out flag) and possibly legal issues to consider - ie we have always followed the 'once they opt out a given email address they are opted out' philosophy and changing that could have broader implications.

@Eileenmcnaughton The issue is 100% with this scenario 'If 2 is the problem you are trying to solve then that makes sense & we can look to address that in this phab', thanks for agreeing with us!

@SHust Hi Sandra, our current logic for is_opt_out to silverpop is either conctact have is_opt_out true, mark it true so the only situation that bother you is when
contacts have different email addresses, and the latest one is is not opt out, while the old contact opt_out
as the ui here

Screenshot 2023-11-02 at 5.34.31 PM.png (848×2 px, 187 KB)

and you would like to mark it as no, so instead of we update the civi core or add a complex resolver to solve this situation, we have a easy approach which is to make the is_opt_out editable on the deduper list page,
so if those situation is not that much, what we can do is uncheck the is_opt_out from the old contact, save it and then merge it.
then you will have a merged contact with is_opt_out as false.

Is that sounds like a solution for ya?

Change 967066 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] add is_opt_id option as to deduper ui with ability to edit

https://gerrit.wikimedia.org/r/967066

@AnnWF Thanks for making the opt-out editable, it will definitely do the trick we desperately need! Is there a way to add the same 'uncheck' box to the legacy interface?
{F41459195}

XenoRyet set Final Story Points to 4.

@AnnWF Thanks for making the opt-out editable, it will definitely do the trick we desperately need! Is there a way to add the same 'uncheck' box to the legacy interface?
{F41459195}

Hi @SHust , glad this helps, and what do you mean by legacy interface? btw, you will see the checkbox from manual merge UI too once that upstream bug fix merged.

@AnnWF Legacy interface is what we call the manual merge UI, so we're all set. Thanks again for making this happen :)

Change 974723 had a related patch set uploaded (by Wfan; author: Wfan):

[wikimedia/fundraising/crm@master] Update CRM deduper merger.php

https://gerrit.wikimedia.org/r/974723

Change 974723 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Update CRM deduper merger.php

https://gerrit.wikimedia.org/r/974723