Determine what was breaking GlobalCollect in paymentswiki 3730cff20
Closed, ResolvedPublic2 Story Points
Actions

Description

Was "unbreak now", but we've rolled back the bad change.

It looks like GC donors are being sent to the error page after a good transaction. See order_id 8274273886 and Civi IDs 12107504 and 12107844.

Nov 10 21:57:15 X globalcollect_gateway_trxn: 29160570:8274273886 "2015-11-10T21:57:15+00:00" "29160570" "Gateway response: Original Response Status (pre-SET_PAYMENT): 600" "false" 
Nov 10 21:57:15 X globalcollect_gateway: 29160570:8274273886 Pushing transaction to queue [complete]
Nov 10 21:57:15 X globalcollect_gateway: 29160570:8274273886 Clock at Confirm_CreditCard: 1.2096 (1447192635.9241)
Nov 10 21:57:15 X globalcollect_gateway: 29160570:8274273886 Displaying thank you page for final status complete
Nov 10 21:57:15 X globalcollect_gateway: 29160570:8274273886 Redirecting to [/index.php?title=Special:GlobalCollectGateway&appeal=JimmyQuote&ffname=error-cc&gateway=globalcollect&country=US&currency_code=USD&amount=25&language=en&utm_source=sp49981691.default%7Edefault%7Edefault%7Edefault%7Econtrol.cc&utm_medium=email&utm_campaign=C15_Email1&payment_method=cc]
Nov 10 21:57:20 X globalcollect_gateway: 29160597:9157669488 Attempting to set a valid form for the combination: {"amount":"25.00","ffname":"cc-vmad","country":"US","currency_code":"USD","payment_method":"cc","payment_submethod":"","recurring":"","gateway":"globalcollect","utm_source":"sp49981691.default~default~default~default~control.cc","referrer":"https:\/\/payments.wikimedia.org\/index.php\/Special:GlobalCollectGatewayResult?order_id=8274273886&amp;REF=000000657082742738860000100001&amp;RETURNMAC=y1DR%2F1BMpdFI3SwMtGs%2Fzi24ZcUE0FSNzzlgOie6QNQ%3D&amp;AVSRESULT=Z&amp;CVVRESULT=M&amp;EXTERNALREFERENCE=29160570.14300"}

Details

Related Gerrit Patches:

mediawiki/extensions/DonationInterface : deployment	Return from function after redirect
mediawiki/extensions/DonationInterface : master	Revert logic change from logging enhancement
mediawiki/extensions/DonationInterface : master	Return from function after redirect

Related Objects
Search...

Status	Assigned	Task
Open	None	T112923 [EPIC] Banner history post launch
Resolved	AndyRussG	T112022 Associate banner history ID with contribution_tracking ID
Resolved	Ejegg	T118349 Determine what was breaking GlobalCollect in paymentswiki 3730cff20

Event Timeline

awight created this task.Nov 10 2015, 10:57 PM

awight raised the priority of this task from to High.

awight updated the task description. (Show Details)

awight added projects: FR-Ingenico, MediaWiki-extensions-DonationInterface, Fundraising-Backlog, Fundraising Sprint X-Ray Spex, Unplanned-Sprint-Work.

awight added subscribers: awight, Ejegg, MBeat33.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 10 2015, 10:57 PM

Possible culprit: I added a redirect to the fail page in https://gerrit.wikimedia.org/r/#/c/250062/2/globalcollect_gateway/globalcollect_resultswitcher.body.php,unified

I thought it was plugging a hole where the existing code left the user stranded after an error, but that might be wrong. Maybe the redirect to the thank you page needs a 'return' after it?

Just noting that the offending change (maybe this or another) is still on our deployment branch. We need to either revert or solve this problem, before we can deploy anything else... The bug looks pretty simple, as long as we can reproduce it locally.

It looks like all the affected donors were from 11/10, but we will keep an eye out for more. We refunded 61 duplicate transactions so far, and these tickets are really DS labor intensive - when donors get error messages they tend to make multiple attempts, and when they find out they've all processed are understandably concerned. I would suggest this be put back to Unbreak Now if possible - another batch of these really would impede our capacity to handle donor tickets quickly.

DStrine moved this task from Triage to Closed Sprint Work Completed in Q2 1516 on the Fundraising-Backlog board.Nov 12 2015, 8:58 PM

AndyRussG added a parent task: T112022: Associate banner history ID with contribution_tracking ID.Nov 14 2015, 12:06 AM

MBeat33 mentioned this in T118611: Zendesk accounts for fr-techsen.Nov 16 2015, 11:56 PM

AndyRussG added a subscriber: AndyRussG.Nov 17 2015, 6:10 PM

awight set Security to None.Nov 17 2015, 6:17 PM

awight edited a custom field.

The paymentswiki deployment branch has been reverted, this is still a priority to fix however.

AndyRussG claimed this task.Nov 18 2015, 2:12 PM

AndyRussG moved this task from Backlog to Doing on the Fundraising Sprint X-Ray Spex board.Nov 19 2015, 1:48 AM

Change 254181 had a related patch set uploaded (by Ejegg):
Revert logic change from logging enhancement

https://gerrit.wikimedia.org/r/254181

gerritbot added a project: Patch-For-Review.Nov 19 2015, 7:39 PM

Change 254633 had a related patch set uploaded (by Ejegg):
Return from function after redirect

https://gerrit.wikimedia.org/r/254633

Now 99% sure that fallthrough displayFailPage was the culprit. Basically, we're calling redirect with the thankyou page, then calling redirect with the fail page. Calling redirect sets the location header, but it doesn't stop execution of code. When you call it twice, the second call wins. Either of my patches above would fix the problem.

Change 254633 merged by jenkins-bot:
Return from function after redirect

https://gerrit.wikimedia.org/r/254633

Change 254181 abandoned by Ejegg:
Revert logic change from logging enhancement

Reason:
Superceded

https://gerrit.wikimedia.org/r/254181

ReleaseTaggerBot added a project: MW-1.27-release (WMF-deploy-2015-12-08_(1.27.0-wmf.8)).Nov 23 2015, 4:00 PM

AndyRussG reassigned this task from AndyRussG to Ejegg.Nov 23 2015, 8:23 PM

AndyRussG moved this task from Doing to Pending Deployment on the Fundraising Sprint X-Ray Spex board.

Change 254920 had a related patch set uploaded (by Ejegg):
Return from function after redirect

https://gerrit.wikimedia.org/r/254920

Change 254920 merged by Ejegg:
Return from function after redirect

https://gerrit.wikimedia.org/r/254920