Page MenuHomePhabricator

Allow DR to refund transactions from Civi and flag as fraud
Closed, ResolvedPublic

Description

DR currently refunds transactions in the payment processor consoles and then flags them as fraud in a Google Doc.

We would like to have all fraud txns tagged in Civi for e.g. training an ML model and also for proactive FR-tech filter creation.

However, we don't want DR to have to double their refund workload by refunding in a console + then flagging in Civi.

Create a refund workflow in Civi that has a checkbox to flag the donation as fraud. When the box is checked we can store the string 'fraud' in the cancel_reason field on the contributions table.

Event Timeline

From the Civi Docs, the fns to add to CRM_Core_Payment_SmashPig

doRefund function

This is largely similar to doPayment - if you implement this you need to declare that your site supportsRefund (see supports functions).

This function should return an array like return ['refund_status' => 'Completed'];. It can optionaly include trxn_id and trxn_result_code.

https://docs.civicrm.org/dev/en/latest/extensions/payment-processors/paymentclass/

Change #1262205 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/SmashPig@master] Mark Braintree and PayPal with refund interface

https://gerrit.wikimedia.org/r/1262205

Change #1262207 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] WIP support refunds on multiple payment processors

https://gerrit.wikimedia.org/r/1262207

Change #1262277 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] WIP form to submit refunds to processors

https://gerrit.wikimedia.org/r/1262277

Change #1267431 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Refactor SmashPig::doRefund to work with API4

https://gerrit.wikimedia.org/r/1267431

Change #1262207 abandoned by Ejegg:

[wikimedia/fundraising/crm@master] WIP support refunds on multiple payment processors

Reason:

Squashed into Iefced2551cd0a6ed87a8f1003785c86f474d6e4a

https://gerrit.wikimedia.org/r/1262207

Change #1267873 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/SmashPig@master] Set gatewayRefundID on refund responses

https://gerrit.wikimedia.org/r/1267873

Change #1268279 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Add contribution status to context for civicrm_links hook

https://gerrit.wikimedia.org/r/1268279

Change #1267431 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Refactor SmashPig::doRefund to work with API4

https://gerrit.wikimedia.org/r/1267431

Change #1262205 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Mark Braintree and PayPal with refund interface

https://gerrit.wikimedia.org/r/1262205

Change #1262277 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Form to submit refunds to processors

https://gerrit.wikimedia.org/r/1262277

We have an initial version of this deployed (in a contact's contribution list, a new 'Refund' link in the 'More' actions list), but we are working on a version that should support batch refunds from searches.

The UI may change, so don't get too attached to what it looks like now!

Change #1268279 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add contribution status to context for civicrm_links hook

https://gerrit.wikimedia.org/r/1268279

Change #1269005 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] WIP searchTask version of refund form

https://gerrit.wikimedia.org/r/1269005

Change #1267873 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Set gatewayRefundID on refund responses

https://gerrit.wikimedia.org/r/1267873

Change #1269005 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] SearchTask version of refund form

https://gerrit.wikimedia.org/r/1269005

This is deployed, but refunding more than 5 (via coworker) is currently failing with permissions errors.

Change #1286368 had a related patch set uploaded (by Lars SG; author: Lars SG):

[wikimedia/fundraising/crm@master] RunAs user who queued the refund when processing queued refunds

https://gerrit.wikimedia.org/r/1286368

Change #1286368 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] RunAs user who queued the refund when processing queued refunds

https://gerrit.wikimedia.org/r/1286368

Just found another issue with the deployed code while trying to test Lars' coworker patch: when I select 6 contributions from searchkit results and select 'refund' it posts the id list but the form doesn't load. It kicks off this killer query (note no filters)

SELECT civicrm_contribution.id, contact_a.id as contact_id, civicrm_contribution.id as contribution_id, civicrm_contribution.is_template as is_template FROM civicrm_contact contact_a LEFT JOIN civicrm_contribution ON civicrm_contribution.contact_id = contact_a.id WHERE 1 AND ( 1 ) AND (contact_a.is_deleted = 0) GROUP BY civicrm_contribution.id ORDER BY contact_a.sort_name ASC, contact_a.id

Lars' patch to fix the permissions errors under coworker seems to be working, but we're getting new errors under coworker.

For Adyen: Configuration key 'accounts//ws-api-key' does not exist.
For PayPal: POST request to https://api.paypal.com/nvp failed 7 times.
For Gravy: Key cannot be empty

Seems it's somehow not loading the SmashPig context correctly, but it's hard to understand how.

https://phabricator.wikimedia.org/T405053 has advice for running coworker with debug.

Change #1286471 had a related patch set uploaded (by Lars SG; author: Lars SG):

[wikimedia/fundraising/crm@master] Load ids from SK for refund form before core tries to load all contributions

https://gerrit.wikimedia.org/r/1286471

Change #1286471 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Load ids from SK for refund form before core tries to load all contributions

https://gerrit.wikimedia.org/r/1286471

Change #1287451 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Change max sync refunds to setting

https://gerrit.wikimedia.org/r/1287451

Change #1287451 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Change max sync refunds to setting

https://gerrit.wikimedia.org/r/1287451

OK, this is ready to use. The one caveat is that if you try to refund more than 25 donations in a single batch, it will send them to the background job runner which we haven't quite gotten working for refunds in production.

Michael noted that it would be useful to batch refund all donations from an unintended recurring (or optionally all but the first). I filed T426350: Batch refund action for unintended recurring contribution to implement that.

XenoRyet set Final Story Points to 4.

@Ejegg this is really a helpful tool, thank you. Some followup questions:

  • Just to confirm, this works across all payment methods? Adyen and Braintree tested fine, just want to confirm for PayPal direct.
  • Could the refund confirmation screen be made to include the shorter merchant reference #? This would save us an extra step of lookup when providing confirmation to donors - we don't use the long Gravy IDs in that context.
  • Can we add an indicator in Civi as to which agent performed the refund? @AMJohnson checked and didn't see any breadcrumb trail in the Change or Activities logs - this info can be helpful when solving complex tickets.
  • Can the =fraud checkbox be undone or edited after the fact? I'm going to advise the DR team to be rigorous about designating things as fraud (i.e. if it’s likely just a forgotten or unintended donation (email is UTM source, etc) and the donor says it’s fraud, do not apply the check). This is low priority, just curious.

Yes, it should work for Adyen, Braintree, dLocal, Gravy and PayPal direct.

Sure, we can show the merchant reference instead. Would you like it to just replace the processor transaction ID, or is that ID useful too?

Huh, I guess I assumed Civi core code would add an activity for a contribution refund. I think the volume shouldn't be enough to worry about bloating the activity table, so I'll see what we need to do to get those.

Yes, you can edit it - the checkbox just sets the cancel_reason field to 'fraud'. That's the "Cancellation / Refund Reason" box midway down the page on the edit modal.

Thanks, @Ejegg for confirming editability and all the available methods.

If it's possible to display both the merchant reference number and the transaction ID, that would help us keep track in complex tickets where there's a bunch of refunds.

Thanks for adding an activity to the refund process, knowing who did the refunds will be really helpful.

Change #1290825 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Add refund activity from new refund form

https://gerrit.wikimedia.org/r/1290825

Demo-ed this at the DR team meeting today, and this was welcomed with much delight and wiki-love.

...Civi core code would add an activity for a contribution refund. I think the volume shouldn't be enough to worry about bloating the activity table, so I'll see what we need to do to get those.

If adding this Activity is what will enable agents to tell who performed the refund, that would be great. We're aiming to switch over to refunding primarily from Civi when we deprecate the Master Refunds spreadsheet on July 1, and if we can have that visibility by then that would be excellent.

Reopening this Task per prior comment

Change #1291025 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Add invoice ID to refund contribution form

https://gerrit.wikimedia.org/r/1291025

Change #1290825 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add refund activity from new refund form

https://gerrit.wikimedia.org/r/1290825

Change #1291025 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add invoice ID to refund contribution form

https://gerrit.wikimedia.org/r/1291025

OK, the merchant reference is now appearing on the refund form, and it's now creating an activity with the person doing the refund as the 'Added By' contact.

The activity should list merchant reference and txn ID too, but the merchant reference was blank on my test activity - I'm looking into why that happened. https://civicrm.wikimedia.org/civicrm/activity?atype=80&action=view&reset=1&id=278999576&cid=2241615&context=activity&searchContext=activity

Thank you @Ejegg Where does the Added By info live? I recently refunded CID 70442458 and don't see the refund in Activities, how would another agent be able to tell it was me?

Odd, you did that about 10 minutes after I refunded my test donation. I got an activity on the activity tab and that donor didn't. Let me see what's happening there.

Change #1293804 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] Fix typo in refund param select list

https://gerrit.wikimedia.org/r/1293804

Change #1293804 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Fix typo in refund param select list

https://gerrit.wikimedia.org/r/1293804

OK, that other little fix is deployed. I am now seeing the refund activities under the donor's activity tab, e.g. this one I just refunded: https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=70449310

Excellent, thank you @Ejegg !