Page MenuHomePhabricator

Silverpop data inaccurate - two donors with the same contact ID
Closed, InvalidPublic1 Estimated Story Points


During a spot check today, I found another instance of inaccurate data in Silverpop. I was searching for CID = 244698, and curiously, Silverpop returned two contacts for the same contact ID. Their email addresses and names were different, but all their contribution history was the same. One was actually CID 244698, and the other was CID 6962521 (or related; this donor has 3 unmerged records).

This seems like it could potentially be a big problem, if we are confirming inaccurate donation data to past donors. It also suggests we may not be filtering out recent donors due to having the wrong contribution history. It's also possible this is the only instance of the problem, I have no clue. @MBeat33 hasn't noticed any donor complaints, but Donor Services is working through a mountain of emails right now.

Can tech look into the extent of the problem so I can decide if we need to stop sending email? I have 1.5m emails to get out in the next 3 days, so skipping tomorrow would be a bummer.

Event Timeline

CCogdill_WMF raised the priority of this task from to Unbreak Now!.
CCogdill_WMF updated the task description. (Show Details)
CCogdill_WMF added subscribers: CCogdill_WMF, MBeat33.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 9 2015, 1:10 AM
Ejegg added a subscriber: Ejegg.Dec 9 2015, 1:48 AM

@CCogdill_WMF, I see 325 duplicated contact IDs in the export. I'll keep digging for the cause and a solution.

Thank you, @Ejegg, 325 is a comforting number. I will proceed with sending
emails for now. I'm tempted to say we can deescalate priority unless you
think this is a sign of a greater issue. Crazy that I happened to pick 1 of
325 when doing a random spot check!

Le mardi 8 décembre 2015, Ejegg <> a
écrit :

Ejegg added a subscriber: Ejegg.
Ejegg added a comment.

@CCogdill_WMF, I see 325 duplicated contact IDs in the export. I'll keep
digging for the cause and a solution.



To: Ejegg
Cc: Ejegg, atgo, MBeat33, CCogdill_WMF, Aklapper, DStrine, cwdent

atgo lowered the priority of this task from Unbreak Now! to High.Dec 9 2015, 6:24 PM
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Zapp board.Dec 23 2015, 10:23 PM
Ejegg set Security to None.Dec 29 2015, 8:08 PM
Ejegg edited a custom field.
DStrine assigned this task to Ejegg.Dec 29 2015, 8:13 PM
Ejegg added a comment.Dec 29 2015, 9:28 PM

@CCogdill_WMF, the only duplicated contact IDs I see in the export now are Civi contacts with more than one email address (most probably merged at some point).

I don't see anyone with the contact IDs you mentioned above in the export.

Should we be limiting the exports to email addresses where is_primary = 1? I guess we should still include non-primary ones in the unsubscribe export.

Okay, maybe the data issue resolved. I'm fine to assume that since it was such a low number to begin with.

I assumed we were always using is_primary = 1. @awight always reminds me to include that parameter when I'm pulling one-off email lists,

atgo closed this task as Invalid.Jan 5 2016, 10:11 PM

I'm closing this for now... @CCogdill_WMF if you run into it again let us know for sure.

@atgo I am a bit worried if we are passing over non-primary emails. Does that mean than I may email the same contact record twice with two email addresses, or did the check @Ejegg just ran rule out that possibility?

mmodell removed a subscriber: awight.Jun 22 2017, 9:42 PM