Page MenuHomePhabricator

New Segmentation Framework not quite usable yet
Closed, ResolvedPublic

Description

I ran a query in Acoustic for New donors (status code 10) and a quick search revealed that there were donors pulled in the query that had a first_donation_date before the fiscal year FY2324, which shouldn't be possible based on the definition.

Definitions here: https://civicrm.wikimedia.org/civicrm/wmf-segment

When looking closer, the pattern is that the ones I found were duplicates in Civi that haven't been merged yet. However, the volume is enough to where I don't feel comfortable using the framework alone to email people; there would be people getting emails that aren't eligible.

My guess is that the reason this is happening is because Civi has the segmentation framework keyed to CID and Acoustic is keyed for Email Address. (and Eileen just confirmed and agreed in Slack).

Can we find a solution fairly quickly on this? We're starting to implement the framework codes more regularly so I think this should be flagged high priority. Thanks

Event Timeline

Here's a file of CIDs to look at prioritizing dedupe / triaging how we can fix this long term with fr-tech.

@Eileenmcnaughton @SHust

It's everyone in Italy (language = it) eligible for the upcoming campaign
that is status code 10 "new"
that has a first_donation_date prior to July 1, 2023 (which shouldn't be possible in this definition).
It gave me 2457 names

deleting my last comment above since it's a different issue; new ticket here: T365523

Also noting other bugs with the segmentation framework, this is another one:
T363959

This is the same issue as the non-donor one - https://phabricator.wikimedia.org/T363959#9819404 - when we 'Acoustic-merge' a contact (that may be a non-duplicate or a not-yet-merged contact) we are using MAX() to get the status - because of the way the recurring ones are higher priority but are not numbered in such a way we can use MIN - it would be better to fix up that numbering - if there are no blockers

Actually this is not quite the same issue - we calculate the Donor Statuses based on the donations they have in CiviCRM. It's a pretty complex calculation & when we update to Acoustic they just get the highest priority status from the contacts being 'Acoustic Merged' - the New status is higher priority that the Current status so that gets exported.

This is somewhat mitigated by the dedupe team doing a clean up on upcoming countries

jgleeson moved this task from In Progress to Backlog on the Fundraising Tech - Chaos Crew board.
jgleeson subscribed.

@KHaggard we have addressed a couple of issues since this was logged - what is the status on this phab?

KHaggard claimed this task.

Yeah I think this is pretty much wrapped up now. The only thing we're working on now is the database mapping and then I'm going to do another QA check after we do a full db file push that everything's sorted correctly. But I suppose that could be a new task if needed, if anything else comes up.

Going to resolve this now, but feel free to re-open if we want to keep using it for any reason. @Eileenmcnaughton