Page MenuHomePhabricator
Create Task
Maniphest T107845

Spike: Lots of 21000050 errors for Globalcollect, since July
Closed, ResolvedPublic1 Estimated Story Points
Actions

Description

What is this about?

orphans: globalcollect_gateway_tr 27169358:4479613483 processResponse Error 21000050 : Blocking validation problems with this payment. Investigation required!

We received 20 of these today, from the frontend and the orphan slayer. Grepping through the logs, I see that we got 1,326 of these in July, 111 in June, and only 2 in May. I don't like that it's growing bigger. Doesn't look like it's proportional to traffic.

Also: We seem to be in danger of losing some institutional knowledge, let's document the common errors and investigate what constitutes a "flood of errors" for each one. I've never heard of this error, but as recently as a year ago someone lovingly crafted code paying homage to this particular error.

Details

ProjectBranchLines +/-Subject
mediawiki/extensions/DonationInterfacemaster+2 -1Log real GC error on validation problem
mediawiki/extensions/DonationInterfacemaster+2 -1Log original GlobalCollect validation error
Customize query in gerrit

Related Objects

Event Timeline

awight created this task.Aug 4 2015, 5:58 AM
awight raised the priority of this task from to High.
awight updated the task description. (Show Details)
awight added projects: Fundraising-Backlog, MediaWiki-extensions-DonationInterface.
awight added a subscriber: awight.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 4 2015, 5:58 AM
DStrine moved this task from Triage to Sprint C Feb 2nd to Feb 16th on the Fundraising-Backlog board.Aug 11 2015, 9:36 PM
• atgo set Security to None.Aug 12 2015, 10:30 PM
• atgo edited a custom field.
gerritbot added a subscriber: gerritbot.Aug 12 2015, 10:42 PM

Change 231186 had a related patch set uploaded (by Ejegg):
Log original GlobalCollect validation error

https://gerrit.wikimedia.org/r/231186

gerritbot added a project: Patch-For-Review.Aug 12 2015, 10:42 PM
gerritbot added a comment.Aug 13 2015, 5:26 PM

Change 231186 merged by jenkins-bot:
Log original GlobalCollect validation error

https://gerrit.wikimedia.org/r/231186

Ejegg mentioned this in rEDOI53823713af8a: Log original GlobalCollect validation error.Aug 13 2015, 5:26 PM
Ejegg mentioned this in rMEXTe911e9f7861f: Updated mediawiki/extensions Project: mediawiki/extensions/DonationInterface….
• atgo added subscribers: Ejegg, • atgo.Aug 19 2015, 8:50 PM

@Ejegg do the patches on this mean that this is either in review or deployed? Should we add Fundraising Sprint Queen ?

Ejegg added a comment.Aug 19 2015, 8:52 PM

Atgo, the patch just lets us catch more information, and was deployed on Thursday. I'll see if it's caught any of these yet.

• atgo added a comment.Aug 19 2015, 8:53 PM

ah ok. just doing some cleanup in advance of Prawning, so no hurry.

DStrine moved this task from Sprint C Feb 2nd to Feb 16th to Closed Sprint Work Q1 1516 on the Fundraising-Backlog board.Aug 19 2015, 10:49 PM
• atgo added a project: Fundraising Sprint Rowlf the Dog.Aug 19 2015, 10:58 PM
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Rowlf the Dog board.Aug 20 2015, 4:52 PM

Oops, that last patch was useless. Logging our generic error message instead of the actual error message from GC! Let me fix that...

gerritbot added a comment.Aug 20 2015, 5:05 PM

Change 232761 had a related patch set uploaded (by Ejegg):
Log real GC error on validation problem

https://gerrit.wikimedia.org/r/232761

Ejegg claimed this task.Aug 20 2015, 8:12 PM
gerritbot added a comment.Aug 21 2015, 2:23 AM

Change 232761 merged by jenkins-bot:
Log real GC error on validation problem

https://gerrit.wikimedia.org/r/232761

Ejegg mentioned this in rMEXT0d40a940f4f3: Updated mediawiki/extensions Project: mediawiki/extensions/DonationInterface….Aug 21 2015, 2:23 AM
Ejegg mentioned this in rEDOI964d119acb73: Log real GC error on validation problem.
Ejegg added a comment.Aug 27 2015, 1:50 AM

Caught one! From the Japan form, trying to donate via JCB: '21000050 REQUEST 10380495 VALUE *** OF FIELD CVV IS NOT A NUMBER WITH MINLENGTH 1, MAXLENGTH 4 AND PRECISION 0'

What's up with that? We're not collecting CVV for GlobalCollect.

• atgo added a comment.Aug 27 2015, 2:37 AM

We aren't...? I thought we were.

Ejegg added a comment.Aug 27 2015, 2:42 AM

Nah, for GC that's all in their iframe. But we're even seeing some expiry date validation fails from them, so it looks like they're just screwing up their iframe's client-side validation somehow and still bouncing donors back to our return URL.

awight added a comment.Aug 27 2015, 2:44 AM

Chiming in, we get CVV validation results back from GC, so that might be why it looks like we're collecting the number ourselves.

Ejegg added a project: Fundraising Sprint Snoop (Dogg|Lion).Sep 3 2015, 8:16 PM
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Snoop (Dogg|Lion) board.Sep 3 2015, 8:22 PM
DStrine added a project: Fundraising Sprint Tom Waits.Sep 17 2015, 1:12 AM
Ejegg moved this task from Backlog to Doing on the Fundraising Sprint Tom Waits board.Sep 17 2015, 2:36 PM
DStrine added a project: Fundraising Sprint UB40.Sep 30 2015, 10:59 PM
DStrine moved this task from Closed Sprint Work Q1 1516 to Closed Sprint Work Completed in Q2 1516 on the Fundraising-Backlog board.Oct 5 2015, 5:30 PM
Ejegg added a comment.Oct 14 2015, 9:50 PM

Emailed GC 10/14 asking that they look into improving validation in their iframe.

Ejegg closed this task as Resolved.Oct 14 2015, 9:59 PM
Ejegg moved this task from Backlog to Done on the Fundraising Sprint UB40 board.
DStrine added a project: WorkType-Maintenance.Oct 20 2015, 8:39 PM
DStrine added a project: Fundraising Sprint William Shatner.Oct 28 2015, 10:58 PM
DStrine removed a project: Fundraising Sprint William Shatner.Oct 28 2015, 11:01 PM
DStrine added a project: FR-Ingenico.Nov 19 2015, 6:28 PM
awight added a comment.Mar 29 2016, 11:28 PM

Saw another one of these on March 29, 34622232:1667666430

mmodell removed a subscriber: awight.Jun 22 2017, 9:44 PM
DStrine edited projects, added Fr-tech-archived-from-FY-2015/16; removed Fundraising-Backlog.Jan 11 2018, 9:45 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL