We disabled the audit download and parsing jobs in response to an outage notification from PayPal, and never reenabled. Now we have to get the old data and backfill like crazy.
Description
Description
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | awight | T112920 Backfill from Aug 25-Sept 17 PayPal audit outage | |||
| Resolved | awight | T112989 Parse PayPal audit data from Aug 25-Aug 27 + Sep 16 gaps | |||
| Resolved | awight | T113087 PayPal audit parser is not associating donations with the correct contribution_tracking record | |||
| Resolved | awight | T113503 Backfill missing associations between PayPal audited donations and contribution_tracking |
Event Timeline
Comment Actions
@awight Does this mean PayPal donations have not been recorded during this period - ie. test results might be skewed? thanks!
Comment Actions
Good question. Historically, PayPal comes in almost entirely through the listener, and the audit system is the source for about 0.5% of the donations in Civi. During the period that the IPN listener was also down, however (currently still down), we wouldn't have recorded any donations at all.
Comment Actions
BTW, once the data is backfilled we should be able to re-run test results and get trustworthy stats.
Comment Actions
Waiting for a response from PayPal, currently the situation seems to be that we should not parse the audit files from Aug 25, 26, and 27th. This would be accomplished by moving the files to the archived directory without running the parse job. Hopefully it's actually safe to use that data, however.
Comment Actions
Okay, suspect files are currently waiting in /archive/<host>/audit/paypal/archive/hold-T112920/
Comment Actions
Threw one more broken file in the bucket, SAR-20150916.01.002.CSV -- none of the 93 subscription creations from that file were parseable.