Page MenuHomePhabricator

rework fredge search to include new fields and general debug
Closed, ResolvedPublic

Description

DoD

  • Add these fields to the fredge report Evelyn is using in civi:
    • IP address
    • Country
    • Currency code
    • Order amount
    • “TS” the timestamp in which they first hit payments wiki ( in contribution tracking table )
  • also the scrambling of results earlier might be an indication that the report might break at high amounts of donations submitted. If anyone can spend time looking at this, that would be great but we don't have large lists to work with at the moment.

Original bug report
As part of the daily review of the stopped fraud que, I am pulling all transactions (merchant Reference #'s) stopped at 600 out of Ingenico, sorting Merchant reference #'s, by date, from newest to oldest and submitting them via Civi Fredge to obtain a list of fraud parameters associated with the list I submit. The results I am getting are oddly starting to be randomly scrambled as of 12.2....Coincident to the issue I raised in Phab task https://phabricator.wikimedia.org/T239769 (not sure if it is related at all) which sees erroneous transactions stopped at 600. Please refer to the 600 trx stopped in this spreadsheet : https://docs.google.com/spreadsheets/d/1z8sOMtfgrlb5P0JNKTj_Fq6ba4UCgBo-9EqsLU9yFls/edit#gid=503946220. I've tried to highlight the MR #'s in red that are randomly scrambled. This has not been the case for the last 4 months since I've been driving this process. Is there some reason the Civi query is scrambling the result? The Fredge results are key to decisioning on manually settling transactions or not and the search results appearing out of order protract the time it takes to conduct the review, which is already a high overhead process.

Thanks!

Event Timeline

I think the original ticket relating to this is here for background context

I just pulled the fraud results for yesterday . 12.9 and just wanted to say that the scrambling referred to above is down to a handful of results out of order from the Fredge query. I am wondering if this could have been volume related? The scrambling seemed to be at its worst while we were at peak volumes, but reducing as our volumes are..... I can live with what I've got now... so less of a priority.

To be clear on this. This is still occurring but it has significantly
reduced and is manageable. It would be good to resolve for next peak if
possible.

4.20.2020. Hello, I wanted to update this task for as we are running Big Bundle, NO, AT, BE and SE campaigns right now, I am seeing a resurgence of the Fredge process scrambling results or pulling up blanks on many transactions. This makes researching the fraud results each day difficult and makes the manual process 2-3 times longer than it does when the Fredge data pull process just works. Hoping this can make it into a Sprint before Big English this year. Thanks!

DStrine renamed this task from Fredge search in Civi randomly scrambling search results to rework fredge search to include new fields and general debug.Jul 16 2020, 4:34 PM

Change 614690 had a related patch set uploaded (by AndyRussG; owner: AndyRussG):
[wikimedia/fundraising/crm@master] WIP Add fields to fredge report

https://gerrit.wikimedia.org/r/614690

Change 614690 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Add country and form amount to fredge report

https://gerrit.wikimedia.org/r/614690

Hi! The changes in the task description are now available on production. @EMartin, could you please confirm that this works for you? Thanks so much!!!

@AndyRussG. Yes! It's great! I am forever in your gratitude. I really
appreciate this change.