Gravy Card Payment AVS check triggering false positives.
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jgleeson
	Jun 9 2025, 1:12 PM

Project Tags

Referenced Files

None

Subscribers

Description

After deploying an update to SmashPig today. I tried to complete a live payment to confirm everything still worked as expected. I was rejected four times across three different cards and two different IPs for two different people, Roxy and me.

It felt odd, so I started looking in the logs, and they were all failing due to fraud filter trips, specifically the AVS check coming back as a no match, pushing up my risk score significantly.

I then tried donating via Adyen directly with the payment method I used first, and it completed successfully as expected. Interestingly, my address details did not trip the AVS check on the Adyen payment.

It seems worthwhile to explore, in case we are rejecting valid donors due to the same issue.

Transctions:

Gravy - Jack - authed:rejected (AVS no match) (Postcode 'PR8 6LR')
Gravy - Roxy - authed:rejected. (AVS no match) (Postcode 'PR86LR')
Adyen - Jack - authed:catured (AVS / CVV full match) (Postcode 'PR86LR')

Note: For the first couple of attempts, I was using the standard UK postcode notation of PR8 6LR (with a space between parts), and then I tried removing the space to see if it made a difference, but it didn't for Gravy, they all failed.

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Log and return early if AVS & CVV are empty in RiskScorer.	Jgleeson	wikimedia/fundraising/SmashPig	master	+5 -2

Customize query in gerrit

Event Timeline

jgleeson created this task.Jun 9 2025, 1:12 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 9 2025, 1:12 PM

AKanji-WMF triaged this task as High priority.Jun 9 2025, 8:33 PM

AKanji-WMF moved this task from Triage to Chaos Crew Backlog on the Fundraising-Backlog board.

jgleeson claimed this task.Jun 11 2025, 12:10 PM

jgleeson moved this task from Backlog to In Progress on the Fundraising Tech - Chaos Crew board.

I've found the issue here. In short, Gravy is sending us back incorrect AVS results, likely due to an error in their mapping. I'll write it up tomorrow with an example and reach out to Gravy.

Gravy has been informed of the issue here

jgleeson raised the priority of this task from High to Unbreak Now!.Jun 26 2025, 10:02 AM

Restricted Application added a subscriber: Pcoombe. · View Herald TranscriptJun 26 2025, 10:02 AM

I've redirected both donatewiki and banner cards to adyen

Thanks @Pcoombe!

MBeat33 subscribed.Jun 26 2025, 1:06 PM

Change #1164214 had a related patch set uploaded (by Jgleeson; author: Jgleeson):

[wikimedia/fundraising/SmashPig@master] Log and return early if AVS & CVV are empty in RiskScorer.

https://gerrit.wikimedia.org/r/1164214

gerritbot added a project: Patch-For-Review.Jun 26 2025, 1:53 PM

krobinson subscribed.Jun 30 2025, 2:01 PM

Gravy has provided a report on the affected transactions here.

At a high level, the numbers look like this:

Total affected by AVS mapping issue

6563

However, according to the report, most of these transactions have a current status of 'capture_succeeded' (6381/6563), indicating that we've collected the funds! I need to review some of the transactions to determine how this happened, as I was expecting the majority of these to be stuck at 'authorization_voided' due to us declining them.

The AVS code breakdown looks like this

Adyen AVS result	Prior Adyen Outcome	Gravy AVS result	Current Gravy Outcome	Count
20 Name, address and zip match	full match - process	no match	decline	3599
21 Name and address match, postal code doesn't match	full match - process	no match	decline	27
22 Name matches	partial match - process	no match	decline	1025
23 Postal code matches, name doesn't match	partial match - process	no match	decline	136
24 Both postal code and address match, name doesn't match	partial match - process	no match	decline	1620
25 Address matches, name doesn't match	partial match - process	no match	decline	7
26 Neither postal code, address nor name match	no match - decline	no match	decline	149

The earliest transaction was in December last year.

jgleeson moved this task from In Progress to Blocked on the Fundraising Tech - Chaos Crew board.Jul 1 2025, 8:47 PM

Change #1164214 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Log and return early if AVS & CVV are empty in RiskScorer.

https://gerrit.wikimedia.org/r/1164214

Maintenance_bot removed a project: Patch-For-Review.Jul 3 2025, 1:30 PM

I spent some time today testing out AVS responses through Gr4vy. I have Visa debit and credit cards at my disposal. I tested out the logic in the phab and received the intended results from the mapping on the AVS logic (see AVS test on the Gr4vy tracker). Is this enough to satisfy that the Gr4vy mapping is corrected or do we need to test this out on AMEX as well?
https://docs.google.com/spreadsheets/d/117qd1q2wEGjjP1FtkOkO4RHGu6cYN_Eh9l0kVtAC2qo/edit?gid=31242775#gid=31242775

jgleeson moved this task from Blocked to Being Reviewed on the Fundraising Tech - Chaos Crew board.Jul 14 2025, 7:19 PM

Hi @EMartin. Thanks for helping with the testing. Visa transactions appear to use different AVS codes than the ones related to this bug.
I checked the AVS code returned for your first test, and it was: "avsResult": "7 Both postal code and address match", so it appears we need to use AMEX to invoke the AVS 20-series codes mentioned above.

I went ahead and started with a few of my own AMEX tests, and I successfully invoked AVS 20 and AVS 24, which are now mapping correctly to a match. I performed four tests in total, first testing with Adyen directly to confirm the expected behaviour, before repeating the same test on Gravy.

We now need to trigger the other scenarios, some of which we can perform via the sandbox. Signs so far look good!

The test coverage table now looks like this:

Adyen AVS result	Adyen Outcome	Prior Gravy AVS result	Current Gravy AVS result	Test transaction
20 Name, address and zip match	full match - process	no match	match - success	00bf83f9-dfd1-446d-aeff-87fbbf5e5268
21 Name and address match, postal code doesn't match	full match - process	no match	tbc
22 Name matches	partial match - process	no match	tbc
23 Postal code matches, name doesn't match	partial match - process	no match	tbc
24 Both postal code and address match, name doesn't match	partial match - process	no match	match - success	57613c38-587e-48fb-a5e9-d75f8196a3ef
25 Address matches, name doesn't match	partial match - process	no match	tbc
26 Neither postal code, address nor name match	no match - decline	no match	tbc

EMartin added a comment.Jul 17 2025, 6:23 PM

This comment was removed by EMartin.

@jgleeson @RKumar_WMF I was able to find some AMEX testers and was able to test the AVS results as above. I've captured the results here.. The only difference I see is that in response 26, Gr4vy /Adyen approved a complete no match where in our direct connection, it declined this response.

Thanks @EMartin, that's great!

Thank you Evelyn.

jgleeson moved this task from Being Reviewed to Done on the Fundraising Tech - Chaos Crew board.Aug 4 2025, 1:08 PM

XenoRyet closed this task as Resolved.Aug 11 2025, 7:34 PM

XenoRyet set Final Story Points to 8.

Gravy Card Payment AVS check triggering false positives. Closed, ResolvedPublicActions

Description

Details

Event Timeline

Gravy Card Payment AVS check triggering false positives.
Closed, ResolvedPublic
Actions