Page MenuHomePhabricator

recent Gravy-PayPal contribs without PII in Civi
Closed, ResolvedPublic

Description

CIDtransactiondatenote
690055654400b395-e9ad-497c-afc0-455b11a5bd35Nov 11PII is visible at Gravy (paypal)
6900455560eba7bf-0312-44a2-83fb-ff8657608993Nov 11PII is visible at Gravy (paypal)

Unclear if related to prior tasks like T395375: some Gravy contribs missing PII (PayPal, trustly)

Details

Event Timeline

two more

CIDtransactiondatenote
690610739614c902-6871-46a9-a399-3aef8718cf4d2025-11-17 02:39:48PII is visible at Gravy
6904641264c63a3e-ce13-43b7-b7f9-2abb99a4e7562025-11-15 18:00:50PII is visible at Gravy
XenoRyet moved this task from Triage to Chaos Crew Backlog on the Fundraising-Backlog board.

The first one initially failed due to a db lock error, according to the logs.

[2025-11-12T23:23:55.734282+00:00] civicrm.DEBUG: DB Error: database lock timeout INSERT INTO `civicrm_contribution` (`contact_id` , `financial_type_id` , `payment_instrument_id` , `receive_date` , `total_amount` , `fee_amount` , `net_amount` , `trxn_id` , `invoice_id` , `currency` , `source` , `contribution_recur_id` , `contribution_status_id` , `check_number` , `tax_amount` ) VALUES ( 3217263 ,  9 ,  25 ,  20251112231821 ,  3.10 ,  0.00 ,  3.10 , 'GRAVY 4400b395-e9ad-497c-afc0-455b11a5bd35' , '239928423.1' , 'USD' , 'USD 3.10' ,  NULL ,  1 ,  NULL ,  0 )  [nativecode=1205 ** Lock wait timeout exceeded; try restarting transaction]  #0

I then see in the Gravy audit file sent over the same night, which should have imported it (the PII is present in the audit data but I've removed it here)

gravy_all_transactions_report_2025_11_13-010204.csv:4400b395-e9ad-497c-afc0-455b11a5bd35,239928423.1,capture_succeeded,2025-11-12T23:16:58.955584+00:00,2025-11-12T23:18:21.567546+00:00,2025-11-12T23:18:18.914350+00:00,2025-11-12T23:18:21.540142+00:00,,310,USD,310,0,paypal,,3S098653PT9365213,b4e49424-f463-4d0d-b05a-b427a4d21c95,paypal-paypal,PayPal,,,,,,ecommerce,0,0

Finally, we end up with this contribution saved against an incomplete contact record, which is a problem for two reasons. The first and obvious reason (missing info). Secondly, it turns out that the donor already has a preexisting contact record here, which should have been picked up during the import and the contribution associated with it, but it wasn't.

I'm now trying to work out what imported the incomplete record. I would have guessed the Gravy audit processor. However, the contribution info suggests this might have been pulled in via a maintenance script on SmashPig, possibly the Gravy importer.

I'm making that assumption based on the contribution source data here:

**Source Name**:	SmashPig
**Source Type**:	job-runner
**Source Host**:	civi1002

I don't think the audit processes run as a job-runner, but I need to confirm that.

one more

CIDtransactiondatenote
69132746e7a96792-79eb-481b-941b-1299678beeacNovember 23, 2025 at 1:31:21PII is visible at Gravy (paypal)

Change #1210653 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/SmashPig@master] Fill more missing fields on Gravy IPNs

https://gerrit.wikimedia.org/r/1210653

Secondly, it turns out that the donor already has a preexisting contact record here, which should have been picked up during the import and the contribution associated with it, but it wasn't.

Turns out this is just a secondary symptom of the donations queue message from the IPN not having any name & email address. Without those, there's no way the donations queue consumer can find the existing contact and dedupe.

Change #1210653 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Fill more missing fields on Gravy IPNs

https://gerrit.wikimedia.org/r/1210653

The working theory here is that both of these transactions failed to be imported the first time around, likely due to database locks during the import run. Usually, when something like this happens, we can recover the transaction via the webhook notifications or the nightly audit. In this case, the webhook processor pulled in the captured donation. However, due to gaps in the PayPal webhook data, the PII was missing, resulting in the CiviCRM contact being created with incomplete data. @Ejegg's related patch plugs that gap so that this shouldn't happen in the future.

The two affected records in the description can be manually updated in CiviCRM using the PII from the Gravy dashboard. I'll do that today, and we can close this one out.

I've manually updated the affected records with the PII from the Gravy console, so I'll move this one to done.

XenoRyet set Final Story Points to 4.