Page MenuHomePhabricator

Donations message with empty body!
Closed, ResolvedPublic2 Story Points

Description

globalcollect-1816361820 had the following headers but no body. Note the "redelivered" flag.

20:31:36              [redelivered] => true
20:31:36              [gateway] => globalcollect
20:31:36              [source_enqueued_time] => 1468613800
20:31:36              [message-id] =>
20:31:36  ID:silicon-54591-1463622100074-2:1676325:-1:1:1
20:31:36              [source_type] => payments
20:31:36              [source_host] => payments1003
20:31:36              [destination] => /queue/donations
20:31:36              [timestamp] => 1468613800058
20:31:36              [expires] => 0
20:31:36              [persistent] => true
20:31:36              [source_run_id] => 74718
20:31:36              [priority] => 4
20:31:36              [php-message-class] =>
20:31:36  SmashPig\CrmLink\Messages\DonationInterfaceMessage
20:31:36              [source_name] => DonationInterface
20:31:36              [source_version] =>
20:31:36  8bf6e911eb43a2d369bf656f07d1b51be0a54f6c
20:31:36              [correlation-id] => globalcollect-1816361820

Details

Related Gerrit Patches:
mediawiki/extensions/DonationInterface : deploymentForce bad encodings to approximate UTF-8
mediawiki/extensions/DonationInterface : masterForce bad encodings to approximate UTF-8
mediawiki/extensions/DonationInterface : master[WIP] Reproduce Ingenico name encoding bug
mediawiki/extensions/DonationInterface : deploymentLog json_encode errors
mediawiki/extensions/DonationInterface : masterLog json_encode errors
mediawiki/extensions/DonationInterface : deploymentLog completed donations
mediawiki/extensions/DonationInterface : masterLog completed donations
wikimedia/fundraising/crm : masterFail if the message body is missing

Event Timeline

awight created this task.Jul 15 2016, 8:38 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald TranscriptJul 15 2016, 8:38 PM

We should test for this condition and throw a reasonable error.

Another one, this was not redelivered:

20:41:47      [headers] => Array
20:41:47          (
20:41:47              [gateway] => globalcollect
20:41:47              [source_enqueued_time] => 1468613976
20:41:47              [message-id] =>
20:41:47  ID:silicon-54591-1463622100074-2:1676475:-1:1:1
20:41:47              [source_type] => payments
20:41:47              [source_host] => payments1003
20:41:47              [destination] => /queue/donations
20:41:47              [timestamp] => 1468613976938
20:41:47              [expires] => 0
20:41:47              [persistent] => true
20:41:47              [source_run_id] => 26984
20:41:47              [priority] => 4
20:41:47              [php-message-class] =>
20:41:47  SmashPig\CrmLink\Messages\DonationInterfaceMessage
20:41:47              [source_name] => DonationInterface
20:41:47              [source_version] =>
20:41:47  8bf6e911eb43a2d369bf656f07d1b51be0a54f6c
20:41:47              [correlation-id] => globalcollect-8604182787
20:41:47          )

Change 299261 had a related patch set uploaded (by Awight):
Fail if the message body is missing

https://gerrit.wikimedia.org/r/299261

awight triaged this task as Unbreak Now! priority.Jul 15 2016, 8:49 PM
Restricted Application added subscribers: Luke081515, TerraCodes. · View Herald TranscriptJul 15 2016, 8:49 PM

Change 299261 merged by jenkins-bot:
Fail if the message body is missing

https://gerrit.wikimedia.org/r/299261

The robust consumer patch is deployed now, so future empty bodies will cause a failmail with the message "Bad news, this message was lacking a body."

Ethereal!

Now we need to track down the source.

awight lowered the priority of this task from Unbreak Now! to High.Jul 15 2016, 9:19 PM

I've been using this query to check Ingenico volume:

select count(*) from civicrm_contribution where receive_date > '2016-07-15 20:17:00' and trxn_id like 'GLOBALCOLLECT%';

also--the #signal-to-noise on our failmails is bad. In the future, we should be able to acknowledge the alarm and have a light quietly flash red...

Change 299280 had a related patch set uploaded (by Awight):
Log completed donations

https://gerrit.wikimedia.org/r/299280

awight raised the priority of this task from High to Unbreak Now!.Jul 16 2016, 10:30 AM

This patch could use some review... I've included a test to make it more palatable... https://gerrit.wikimedia.org/r/299280

Change 299280 merged by jenkins-bot:
Log completed donations

https://gerrit.wikimedia.org/r/299280

Change 299290 had a related patch set uploaded (by Awight):
Log completed donations

https://gerrit.wikimedia.org/r/299290

Change 299290 merged by jenkins-bot:
Log completed donations

https://gerrit.wikimedia.org/r/299290

awight added a comment.EditedJul 16 2016, 7:14 PM

Well. The debug logging just captured:

Jul 16 18:55:37 payments1002 globalcollect_gateway: 37122334:3271638092 Completed donation:

With no data encoded by json_encode( $this->getData_Unstaged_Escaped() ). Not what I was expecting...

I think that means json_encode is actually failing to encode, because any other type of empty data would result in a string like "false".

Change 299297 had a related patch set uploaded (by Awight):
[WIP] Log json_encode errors

https://gerrit.wikimedia.org/r/299297

@MBeat33
I'm realizing that we can detect this error while the donor is still logged in--
Maybe we should display a thank-you page that carefully alerts the donor that *you have paid $XX.00*, but we are experiencing a short delay in getting you a confirmation and welcome letter thanking you for the donation. Please save a copy of this page for your records...

Change 299297 merged by jenkins-bot:
Log json_encode errors

https://gerrit.wikimedia.org/r/299297

Change 299302 had a related patch set uploaded (by Awight):
Log json_encode errors

https://gerrit.wikimedia.org/r/299302

Change 299302 merged by jenkins-bot:
Log json_encode errors

https://gerrit.wikimedia.org/r/299302

Change 299350 had a related patch set uploaded (by Awight):
[WIP] Reproduce Ingenico name encoding bug

https://gerrit.wikimedia.org/r/299350

Change 299352 had a related patch set uploaded (by Awight):
Force bad encodings to approximate UTF-8

https://gerrit.wikimedia.org/r/299352

Change 299350 abandoned by Awight:
[WIP] Reproduce Ingenico name encoding bug

https://gerrit.wikimedia.org/r/299350

Change 299352 merged by jenkins-bot:
Force bad encodings to approximate UTF-8

https://gerrit.wikimedia.org/r/299352

Change 299612 had a related patch set uploaded (by Ejegg):
Force bad encodings to approximate UTF-8

https://gerrit.wikimedia.org/r/299612

Change 299612 merged by jenkins-bot:
Force bad encodings to approximate UTF-8

https://gerrit.wikimedia.org/r/299612

awight claimed this task.Jul 18 2016, 10:51 PM
DStrine set the point value for this task to 2.Jul 20 2016, 10:41 PM
awight closed this task as Resolved.Aug 17 2016, 10:58 PM
mmodell removed a subscriber: awight.Jun 22 2017, 9:44 PM
Restricted Application added a subscriber: Jay8g. · View Herald TranscriptJun 22 2017, 9:44 PM