Dod:
remove unusual characters from zipcode.
email subject 'Fail Mail (civi1001) run-job: Donations queue consume failed with code 2'
Somehow a donor managed to submit the postal code '112”1' which includes a weird double quote character.
ct_id is 90313536 and it came in 20200902.
The actual failure was during the mysql insert, see civi1001:/srv/archive/civi1001/process-control/donations_queue_consume-20200907-013301.log.civi1001.bz2 for details.
| Subject | Repo | Branch | Lines +/- | |
|---|---|---|---|---|
| Filter out unexpected zip code characters | wikimedia/fundraising/crm | master | +52 -0 |
this happened again today. looks like it was due to an audit file record being processed which might explain how the quote got in.
check out the end of /srv/archive/civi1001/process-control/20200909/donations_queue_consume-20200909-013602.log.civi1001.bz2 for more details
This happened again today so I tracked down the problematic audit file here: civi1001:/var/spool/audit/globalcollect/incoming/wx1.000000657020200909.010241.xml.gz
It's an Ingenico audit row containing the following on line 2794384:
<tns:BillingZip>112”1</tns:BillingZip>
for order id 90313536.1
My current thinking is to just fix this field (remove the quote), rezip it up and manually run the Ingenico audit to see if it fixes things.
This happened again today but this time it wasn't an audit record, it was a donation via the payment forms and the quote made it to Ingenico (in an encoded form I think)
Checkout 90800515 in the payments logs on frlog1001 to see what came in.
Looks like we should consider tightening up the input validation on the zip/postal code field to fix this issue so I'll add a ticket for that work and link it to this one for context.
We hardly ever use postal code, so I think it's OK if we either just filter out invalid chars or just blank the field when it's invalid. We shouldn't let it stop the imports.
Change 632540 had a related patch set uploaded (by Jgleeson; owner: Jgleeson):
[wikimedia/fundraising/crm@master] Filter out unexpected zip code characters
Change 632540 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Filter out unexpected zip code characters