Page MenuHomePhabricator

Contributions imported from gravy audit are in CiviCRM in non-USD currencies
Closed, ResolvedPublic

Description

Started July 31, but mostly Sept 25 and Aug 14.

Searchkit

Thank you emails are still showing the currency the donor donated in so they are okay, this is mainly an issue for stats and the large donation bot.

Event Timeline

Cstone renamed this task from Contributions imported from audit are in CiviCRM in non-USD currencies to Contributions imported from gravy audit are in CiviCRM in non-USD currencies.Sep 26 2025, 5:11 PM

Looks like even the older ones in that searchkit were actually imported to Civi on Sept 25

Turned off the gravy audit to stop more from coming in

Change #1191762 had a related patch set uploaded (by Cstone; author: Cstone):

[wikimedia/fundraising/crm@master] Add non USD currency to settled_currency test

https://gerrit.wikimedia.org/r/1191762

Change #1191764 had a related patch set uploaded (by Wfan; author: Eileen):

[wikimedia/fundraising/crm@master] Disambiguate reporting and settlement currency

https://gerrit.wikimedia.org/r/1191764

Change #1191765 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] API to convert contribution to a different currency

https://gerrit.wikimedia.org/r/1191765

Change #1191784 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/SmashPig@master] Don't add misleading settlement info

https://gerrit.wikimedia.org/r/1191784

Change #1191765 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] API to convert contribution to a different currency

https://gerrit.wikimedia.org/r/1191765

Turns out we're still only parsing the all_transactions report for Gravy, not the settlement report. So we shouldn't even be returning those settlement columns from the parser.

Also, please note that the ExchangeRates::convertContribution API call just merged by @Lars needs some more work to keep the Financial Txn rows looking good. In a local test where I converted a 5.35 CAD txn to 3.93 USD, the internal Civi contribution update logic compared old + new amounts, then added a -1.42 USD financial txn to make it come out 'even'.

I'd be happy making it just convert the financial txn rows in place, but the 'proper' thing to do might be to add a -5.35 CAD txn, then a +3.93 USD one instead.

Change #1191784 merged by jenkins-bot:

[wikimedia/fundraising/SmashPig@master] Don't add misleading settlement info

https://gerrit.wikimedia.org/r/1191784

AKanji-WMF moved this task from Triage to Chaos Crew Backlog on the Fundraising-Backlog board.

Change #1192234 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add settlement_currency test

https://gerrit.wikimedia.org/r/1192234

Change #1192234 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add settlement_currency test

https://gerrit.wikimedia.org/r/1192234

Change #1193439 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@master] WIP fix financial txn rows on contribution convert

https://gerrit.wikimedia.org/r/1193439

Change #1191762 abandoned by Eileen:

[wikimedia/fundraising/crm@master] Add non USD currency to settled_currency test

Reason:

this got sucked in

https://gerrit.wikimedia.org/r/1191762

Change #1193439 merged by Eileen:

[wikimedia/fundraising/crm@master] Fix financial txn rows on contribution convert

https://gerrit.wikimedia.org/r/1193439

Change #1219890 had a related patch set uploaded (by Ejegg; author: Ejegg):

[wikimedia/fundraising/crm@deployment] Fix financial txn rows on contribution convert

https://gerrit.wikimedia.org/r/1219890

Change #1219890 merged by Ejegg:

[wikimedia/fundraising/crm@deployment] Fix financial txn rows on contribution convert

https://gerrit.wikimedia.org/r/1219890

@Eileenmcnaughton thank you for the fix! I deployed it today and figured out how to use chaining to get all the non-USD ones converted over. Unfortunately it seems that there are a lot of non-USD donations which were originally recorded in the db with the correctly converted USD amount, then on these same dates were altered to have the original amount (while keeping the USD currency). They generally have two financial transactions against them and will have a total_amount equal to the original_amount, so I think I'll be able to find them. Unfortunately there seem to be a whole lot of rows on each log_conn_id so I don't think we'll be able to use the revert action against them.

I count 4,496 distinct donations recorded wrong, with the bulk of the inaccuracy from about $600k worth of JPY donations T412025: Japan Recurring Gifts September

select count(*), sum(total_amount), original_currency
from civicrm_contribution c
inner join wmf_contribution_extra x on x.entity_id=c.id
where x.original_currency not in ('USD', 'PAB', 'BMD') --two currencies pegged to dollar
and c.total_amount=x.original_amount
and c.receive_date between '2025-07-01' and '2025-12-01'
group by original_currency;
Damilare set Final Story Points to 4.