To clarify, I think the recurring failure message sends after the third and final processing attempt, without doing any kind of scan for a current active recurring donation linked o the same email address. Is that correct?

That looks to be the case. I looked over the code here and it looks like we don't currently check whether the donor has another active recurring donation on the email address.

Thanks for confirming, @jgleeson

If we can determine why they got the fail message 9 days after the donation failed, that would help direct a response. If it was a glitch that's no big deal, but if we are likely to see more cases like this, maybe we could either tighten up the send time, or (though I hesitate to add complexity) think about adding a check before sending the fail message.

@krobinson noticed that the donor had two recurrings that failed in the same month, so that explains the extra email sent.

Maybe the larger question is how or whether to build the fail message in a way that can handle cases where the same donor has multiple recurring donations? The # of CIDs with multiple active recurring donations might be relevant here.

Would it be a tech-debt heavy ask to create a filtering scan for the fail messages to suppress in case the CID has an active recurring donation?

Thanks for telling us about this @MBeat33! @Eileenmcnaughton This does seem like a good idea. In my head, I'd assumed we'd reach out as soon the failure happened but I suppose someone could have started a new one potentially before that.

cid=1681193 ZD 793633

This donor created a new recurring on Aug 14th, but got the Recur fail message in Aug 31st. We're not seeing this problem at a large scale yet, but it's donor-facing in a way that makes it look like we are not acknowledging their renewal, so any piece of code that could check for an active recurring and cancel the fail message if one exists would be a big help.

There are also enough donors with (intended) multiple recurring donations that it might be good to suppress the fail message if there are any (not just recently-started) active recurring donations for the CID. Would it be possible to create that filter?

We had another example of a confused donor:
CID 14238272 / ZD 796016. Donor created a new recurring on 08/31, replacing old donation that processed each month on 5th. When donation failed on 09/05 they received our follow up email, despite the new donation being in place.

Big +1 to creating a filter so those who already have a new donation in place do not receive these emails, as they are confusing to them, and it feels like we're spamming our most proactive, loyal donors.

What is the process by which they create the new ones? They just do it themselves online? I'm wondering if there is a point in the process when we could identify that there is one that needs cancelling rather than just leave it to fail out.

Yes, these are donors who realise their card is going to expire or has been stolen and the donation will fail/cancel, so they create a new one themselves without ever reaching out to us. A scan prior to sending the automated failure message would resolve this.

cid=1960455 is another donor who received the Civi email on October 1, though their donation processed on September 29th, 2020. This is a potential major donor.

The same Adyen reference # 81908248.4 shows two outcomes:
2020-09-30 01:22:56 PDT REFUSED (800 Contract not found)
2020-10-28 19:26:57 PDT SETTLED
but oddly ref # 81908248.3 shows a SETTLED donation on 2020-09-28 19:29:53 PDT

This is another example of why we need the scan for recent donations to filter before Civi sends. @Eileenmcnaughton do you know the scale of that ask (would it be like a lot of story points?). I am concerned that we are going to see more like this case.

I think it's medium - maybe a 2

Thank you. @DStrine what are the prospects for fr-tech having capacity for this this quarter?

@MBeat33 @krobinson are we happy to skip the failed recurring email if a donor has another separate active recurring donation set-up or do we want to tell someone about it and/or log it in some way?

We would love to skip @jgleeson ! If there is a way to scan and remove those who have an active donation from these automated emails, it would save us a big headache and donor confusion.

Oh - and no need to track them. The idea behind these emails is to just keep them donating - if they already realized and set a new one up themselves, then we don't need to reach out.

Thanks for the info @krobinson :)

Change 642517 had a related patch set uploaded (by Jgleeson; owner: Jgleeson):
[wikimedia/fundraising/crm@master] WIP: Add in check for additional recurring cotributions

https://gerrit.wikimedia.org/r/642517

To Test:

• Submit two recurring donations (same name)
• Run civicrm deduper and merge contacts
• Munge recurring db data so that one record gets processed in the next run of the recurring job and one gets skipped. (play with next_scheduled...)
  • Set the failure_count to 2 on the recurring record intended to be processed (saves doing all this twice)
    • Edit the recurring token of the one that is intended to be processed so that it fails (abc123)
• Comment out the "Two recurring charges within 23 days" check in SmashPigRecurringProcessor.php
• Run drush cvapi job.process_smashpig_recurring
  • Payment should fail as expected

To confirm it worked:

• Recurring record should be changed to cancelled with cancellation reason populated.
• civirm_activity log tbl should NOT have an entry for a recurring failmail

Change 642517 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Suppress recurring payment failure failmail

https://gerrit.wikimedia.org/r/642517

Change 643317 had a related patch set uploaded (by Jgleeson; owner: Jgleeson):
[wikimedia/fundraising/crm@master] Remove unnecessary payment_provider_id filter

https://gerrit.wikimedia.org/r/643317

Change 643317 merged by Ejegg:
[wikimedia/fundraising/crm@master] Remove unnecessary payment_processor_id filter

https://gerrit.wikimedia.org/r/643317