The nightly Silverpop imports generate a report in the ESP which tells me, among other things, how many rows were updated. This number has consistently matched the number of rows in the database - so I would expect somewhere around 7.8 million right now.
I checked those reports this morning after @awight confirmed he had made a change to the file (per T107045) and saw that only 2 million records were updated last night. The night before was ~5.5 million, which is around what it's been for the past couple of weeks. I don't have access to reports older than 11/5, so I'm not sure how long the import has been importing this number of rows.
We (Trilogy and me) are really concerned there is something wrong here, and are wondering if we need to stop sending email. Why were only 2 million records updated last night? Has the size of our database changed?
Can we look at the import file history and see when the row count changed from ~7.8m to ~5.5m?
UPDATE
I'm updated with what I believe is related information. Our email copy currently uses a personalization string which reads:
"you gave %%latest_native_amount%% to keep Wikipedia online..."
This has been working great for us, with no complaints to Donor Services once we got the code right. Today, DS received complaints from donors saying, "I gave X amount last year, why are you saying I only gave Y?" I investigated and the donors are correct - our Silverpop data does not match their Civi record. Examples below:
- CID rMEXT8328610bd3eb: Donor last gave $5, Silverpop says $3 (donor never gave $3)
- CID 8568645: Donor last gave $50, Silverpop says $5 (never gave $5)
- CID 5641016: Donor last gave $75, Silverpop says $3 (never gave $3)
For what it's worth, the CIDs all match the donor emails, and Silverpop does seem to have the most recent CID for all these donors despite each having unmerged records. But the donation amounts are incorrect.
We are putting a hold on sending email until this is fixed. Confirming the previous donation amount was a 20% gain for us; we can't afford to remove it from the email, so we have to stop sending.