Page MenuHomePhabricator

recent Gravy / PayPal donations missing PII in Civi
Closed, ResolvedPublic

Description

We're seeing a trickle of PayPal contribs reaching Civi without donor names or emails. I'm not sure if the ACH banner is relevant but it seemed like a common factor.

CIDDateGravy TransactionUTM infoSuspected Issue
683021702025-08-22 23:29:34711b283f-37d7-40cb-bc17-119b159805a7B2526_082116_en6C_m_p1_lg_pym_ACHinfo.no-LP.paypal + WMF_FR_C2526_en6C_m_0715Chinese characters in resposne data / Country code reportedly C2
683069412025-08-23 09:03:15447a2336-4c8f-475c-bfe4-966f620bf442B2526_082116_en6C_dsk_p2_sm_pym_cnt.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
683066652025-08-23 07:43:39d33db659-40b8-425c-a852-1c789308f8ccB2526_082116_en6C_dsk_p1_lg_pym_cnt.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
683035582025-08-23 03:44:48a0db2187-68fc-41b5-ad10-970ec9184bdaB2526_082116_en6C_dsk_p1_lg_pym_ACHinfo.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
682985692025-08-22 14:01:287b3540aa-b5b1-46ec-86ba-c7d22110a015B2526_082116_en6C_dsk_p1_lg_pym_ACHinfo.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
682976612025-08-22 12:17:46b27309e6-6f29-40c0-8870-9880e8e152b5B2526_082116_en6C_dsk_p1_lg_pym_cnt.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
682924232025-08-22 03:49:242ea62706-e897-4b04-8b2e-41bea0a4d51bB2526_082116_en6C_dsk_p1_lg_pym_ACHinfo.no-LP.paypal + WMF_FR_C2526_en6C_dsk_0715Chinese characters in resposne data / Country code reportedly C2
683105262025-08-24 09:00:447d22ec96-b420-4a22-8203-f34f46ffe1b1spontaneousChinese characters in resposne data / Country code reportedly C2

Event Timeline

And one Venmo:

cid=682995098/228fc07cd5-fa9c-420e-8287-f04109783b21spontaneous

The first two transactions appear to be related to a problem with Chinese Characters. I've raised this with Gravy on Slack here

I've updated the description with a "Suspected Issue" column.

I think the fix on our end to handle this in the future is in T402609: Handle null buyer return for paypal in gravy we can at least put the email if gravy returns nothing

And then another phab with related data T401866: Gravy-PayPal duplicate donation in Civi: what to do

It appears that Gravy has resolved the issue specific to the Chinese Character bug. They reported the fix here, so I tested out one of the non-synced examples from the description, and I now see a buyer info block here

@Cstone, even with the fix, I still like your idea in T402609: Handle null buyer return for paypal in gravy as that gives even more cover in the event that the same bug happens again due to a different set of unexpected data, resulting in an incomplete Gravy response.

@MBeat33, all of the reported affected transactions in the task description have now been successfully synced in the Gravy console, and the buyer info is visible.

It appears that the Venmo transaction is failing due to an unrelated, yet unresolved, issue. I think @Damilare and @Cstone were looking into that. We should probably create a new ticket for that, as I believe it's part of completing the Venmo integration.

It looks like these donations were still recorded in CiviCRM, but without any donor name data. I guess we can manually copy it across in this case, as I don't think we're set up yet to receive partial updates of this nature via automated webhooks from Gravy.

I added emails and names, and sent TY emails for these.

Only one recent PayPal missing PII in recent days, 3845a41f-55fd-4edc-9331-0d44f3ac2b6f has name + email info at Gravy, but the PII did not reach Civi. @jgleeson I cam manually add the details but if there's any way we could tell why this one didn't get picked up that would be great.

Hi @MBeat33. That latest one was affected by the PayPal rate-limiting issue that @Cstone investigated here.

I can see the capture failed from the API event log, although I do see the transaction marked as captured in the Gravy UI. I checked the Audit, and I can see it did come in on the 29th via the PayPal audit, but I still don't see it in CiviCRM. I can't see what went wrong. Is it possible it's been attached to another email?

Do you want us to try and pull this one in manually?

Thank you, Jack, it's at https://civicrm.wikimedia.org/civicrm/contact/view?reset=1&cid=68332359. I'm ok with manually importing the PII, but will wait to do so in case it will obscure any trails.

Ah, @MBeat33, I think we might be looking at different transactions.

I was checking out https://wikimedia.gr4vy.app/merchants/default/transactions/3845a41f-55fd-4edc-9331-0d44f3ac2b6f/overview from:

Only one recent PayPal missing PII in recent days, 3845a41f-55fd-4edc-9331-0d44f3ac2b6f has name + email info at Gravy, but the PII did not reach Civi. @jgleeson I cam manually add the details but if there's any way we could tell why this one didn't get picked up that would be great.

However, the CiviCRM record you linked is associated with https://wikimedia.gr4vy.app/merchants/default/transactions/711b283f-37d7-40cb-bc17-119b159805a7/overview and appears to be one of the transactions affected by the C2 country code issue recently resolved by Gravy. I just synced that record, and it pulled in the PII on the Gravy UI. Unfortunately, that one will also need the PII to be manually copied across.

Whoops, sorry for mixing up the transactions. I will copy in the PII for both, if that makes sense.

CiviGravy
cid=683323593845a41f-55fd-4edc-9331-0d44f3ac2b6f
cid=68302170711b283f-37d7-40cb-bc17-119b159805a7

I don't think we have a CiviCRM record for the first one affected by the rate limiting. You might have uncovered a bigger issue if the transactions affected by the PayPal rate limiting did not get imported via the Audit. I'll do some digging to confirm.

Seilo and Poliane have found recent card donations that did not reach Civi.

CIDGravy transactiondatenote
103189288b79cf90-1a4d-4872-872c-ba1ac3ef9e39Aug 14thsettled Aug 15th
8904803851edc69-22a5-4353-84f9-233795126926Aug 25th6 hour settlement

I have found another recent card donations that did not reach Civi and has now been manually added (as per SHust)

CIDGravy transactiondate
110208720657420-1403-4bde-bb86-9c44a8ce3cc4Aug 15th
CIDGravy trxndate
cid=683919895195c9ea-c2d5-43a0-8009-48611ece1506Sep 5

Donor name and email are visible at Gravy, and status is Captured. I can add the PII manually to Civi but wanted to flag.

CIDGravy trxndate
5104115d05d010a-2d8b-47ed-a06d-6828ae98ec76Sep 2

Transaction not in Civi, I've added this to the Transaction Log in the Tracker

I have found one as well:
CID 17722859 - Gravy trxn: c9197f8a-c11a-455b-9e42-ff5d6d721977 - Date: Sept 3 - Method: Visa ApplePay
I'll be adding this to Civi manually, but wanted to share it here.

@SHust I've just found that one (c9197f8a-c11a-455b-9e42-ff5d6d721977) using our snagged transaction importer, so that should come in shortly.

@MBeat33 I also found d05d010a-2d8b-47ed-a06d-6828ae98ec76 and pulled that one in too.

CIDGravy trxndate
9973589bffb44ad-cfa3-47fa-b75f-112e353b80c6Sep 9trxn not in Civi
685145797c402ac7-07b0-479a-9ea0-d392a3d16665Sep 18PII not in Civi
398527464f89341f-07a6-4f64-9144-4b6f49a208c7Sep 2trxn not in Civi

I've added these to the Transaction Log in the Tracker

Adri just shared one: CID 3561691, Gravy trxn 0092693e-fb4b-48d2-bfeb-49934a37143e, August 14th --> @jgleeson, pls let me know if you need me to pull this one manually. Thank you!

Here's another: CID 2985875, Gravy trxn 30c740a3-2b67-4e26-abca-e5ad23fd35d7, also on August 14th. I'll be waiting to hear if I need to pull it manually or not. Thanks, team.

CIDGravy trxndate
9973589bffb44ad-cfa3-47fa-b75f-112e353b80c6Sep 9trxn not in Civi
685145797c402ac7-07b0-479a-9ea0-d392a3d16665Sep 18PII not in Civi
398527464f89341f-07a6-4f64-9144-4b6f49a208c7Sep 2trxn not in Civi

I've added these to the Transaction Log in the Tracker

Hi @MBeat33

  • #1 has an invalid email address, which I think is why it failed to be imported. I can see it was added either via the nightly audit or manually.
  • #2 This one is strange - the donor's first attempt failed, and then the second attempt succeeded. Unfortunately, our snagged import tool doesn't seem to work with Gravy PayPal donations. I'll look into that :(
  • #3 I pulled this one in here

@SHust I can see you've manually added the two contributions you reported. Sorry I didn't get to them sooner!

No worries, @jgleeson, I was just about to share that I had manually done it 😉

Just an update on these, Eileen's been working with the audit to pull in more donations and we have imported a good amount of backlogged gravy transactions from September. Any new missing ones you see from post 2025-09-24 would be good to treat as a bug / possible audit issue as to why the audit didn't import them correctly.

We are still working through audit files with missing transactions on these days in September:
gravy_settlement_report_2025_09_13-010204.csv
gravy_settlement_report_2025_09_20-010207.csv

One bug we did find T405691: Gravy duplicate transaction with mismatched ids

@Cstone thanks for the update.

Would the snagged transactions not just be ones missing PII? As we had one donor ZD: https://wikimedia.zendesk.com/agent/tickets/1736237 / CID 52425541 whose donation was started on 3rd Sept but didn't hit Civi until 24th Sept (Gravy, via card) and so only just got their receipt, 3 weeks late. Is this the same thing, or a new issue?

Also, do you have an idea of scale - ie how many transactions were missed? Just wondering if we're likely to see donor inquiries and thus need to make specific messaging for this.

Thanks for confirming!

cc @LWadleigh @MBeat33

@krobinson my comment was about transactions that were completely missing from civi, the 3 weeks late one was from the audit cleanup run that was happening over the last two days, both the audit and Jack's snagged transaction method will send thank you emails automatically

I can get a number on how many were imported in this audit clean up.

Donor in 1732154 (no CID) flagged that they aren't being billed for their recurring donation. Their donation see transaction d7d408f6-94e7-4f41-85d4-88c5a58556ab was manually settled back in May and never made it to Civi or billed after.

We've informed the donor that due to a technical error the recurring donation did not process successfully and have invited them to try setting up a new one. As it appears at least the initial donation processed could we get their donation from May in Civi? If imported into Civi as a recurring donation if we could also please ensure it's cancelled as it's been many months without billings and as we've asked them to initiate a new one so we want to ensure they don't end up with 2 recurring donations.

Thanks! :)

I added d7d408f6-94e7-4f41-85d4-88c5a58556ab as a one-time donation to the new CID cid=68559348

Hi Yamini has another recurring one that didn't make it through.

ZD 1736891 / Gravy: c28382b9-6b36-4afd-94fd-8eddd9c9fad5

Donor now wants to cancel the recurring, which we can't do. Can we settle the August donation and cancel future installments?

Once this makes it through we will need to merge the new CID with CID 38151530 (old record) as they used a different email. Thanks!

@krobinson I've found that recurring subscription in the logs from August and pulled it in here

You should be able to close that down now. Let me know if you need anything else.

Two more donations that I manually settled on Wednesday, 10/01, didn't make it to Civi.

  • Transaction id: 455e0c0f-0abc-41ad-af11-b4652490bac1 for ZD ticket 1745812 / CID 68122249
  • Transaction id: 23d71e1a-8c64-45f9-abc4-736ff72449d3 for ZD ticket 1741964 / No CID yet

@MBeat33 is importing these two transactions for me into Civi (thank you!), but still wanted to flag them here.

@Cstone ran the Gravy audit, and these last two got picked up into Civi

I'm gonna mark this one as complete. The original issue was fixed when Gravy added support for the c2 language code. However, since this ticket has become a placeholder for ongoing missing Gravy transactions, we should instead create separate, dedicated tickets to track and record those missing transactions more precisely.

XenoRyet set Final Story Points to 16.