Page MenuHomePhabricator

Rationalise our CiviCRM fields that map to reporting / finance reconcilliation
Closed, ResolvedPublic

Description

I couldn't find our existing phab / phabs on this so opening one here. We have a few fields that intersect to describe donations - but they don't seem to intersect correctly meaning a lot of post-work needs to be done to clean them up.

As I write this I'm looking at our financial types and I am about to disable 16 - 20

idnamelast_used
9Cash2025-10-27 10:18:59
10Stock2025-09-17 12:00:00
11In Kind2024-12-06 13:18:00
14Refund2025-09-29 01:17:58
23Donation2024-11-20 11:37:00
26Endowment Gift2025-09-29 03:09:05
31Recurring Gift2025-09-29 03:21:49
32Recurring Gift - Cash2025-09-29 03:47:35

Inactive

15Chargeback2016-02-11 16:21:17
16reversal2016-02-12 21:00:32
18admin_fraud_reversal2016-02-08 21:20:09
19unauthorized_spoof2016-02-09 17:58:07
20admin_reversal2017-11-30 00:12:00

I'm trying to understand how these are used because in a perfect world they would map to ... financial types as in the GL codes... (each financial type has a GL code although more than one financial type can have the same GL code) - ie we are calling most things 'donations' or 'cash' - which are both meaningless

Anyway - first step - disable the unused ones

Details

Event Timeline

Thanks @Eileenmcnaughton - makes sense to disable if they're not being used, but curious how we're capturing chargeback if that one is unused? Similarly, if we reverse a gift (because it's entered in error) how is this captured?

Re. GL codes- I think these should map to channel rather than financial type. Below are the GL's we've agreed on with Finance:

GL Account (Annual Fund)
Online Recurring
Online Banners
Online Email
Online Direct Mail
Online Other (including SMS, portal, sidebar app)
Major Gifts Restricted
Major Gifts Unrestricted
Chapter Gifts

Endowment:
Major Gifts Restricted
Major Gifts Unrestricted
Email
Banner

So..some of the financial types are being used as channels (recurring & endowment), and I'm not sure if there's any logic to how we're using cash vs. donation.

The obsolete financial types have been disabled on production. Is there more to do for this ticket?

Thanks @ERoden-WMF that helps - I was confused over the presence of Major Gifts in both places. I think the thing that is still confusing me is that if we correctly put 'Major Gifts' in channel we would possibly lose some other data like whether it came from Direct Mail? I'm catching up with Joseph in a couple of hours so will get some extra data from him then

These are the channels - and the GL codes where I know the

CodeChannelSource / Notes
43480Recurring GiftOnline Recurring Contributions
43481BannerOnline Banner Contributions
43482EmailOnline Email Contributions
43483Direct MailOnline Direct Mail Contributions
43484Online Other Contributions
43440Chapter GiftsChapter Gifts
43485Major Gifts – Unrestricted
43428Major Gifts – Restricted
Workplace Giving
Direct Solicitation
Planned Giving
Events
White Mail
SMS
Other Offline

OK - after talking with Joseph here is the data we need to use to calculate the Intact fields

Intact export fieldData required
Account codeIs Major gift + Fund (restrictions) + Channel
Location IDfinancial_type (is Endowment)
Dept IDIs Major gift
GLDIMFUNDINGIs Major gift + Fund(restrictions)

Here is how those inputs add up to generate the Account Code

image.png (442×1 px, 67 KB)

Within CiviCRM we don't actually have Major Gifts donations marked as such. Major Gifts in Fund` is one of the options in Fund but if that is selected then the relevant restriction cannot be selected - which makes it unusable for the purposes of financial integration (It is only set to that for a small number of major gifts donations).

On the analytics side Joseph looks into a bunch of fields to figure out which ones to treat as major gifts. This delivers what people are used to seeing - but it is a workaround for not storing the data properly on ingress and is not transparent.

Since Major gifts seems to be a standalone feature of a donation that can be combined with any other one (restrictions, channel, endowment) @JMando and I concluded we should add a new field is_major_gift to the gift data table. This would be calculated on ingress (& at some point backfilled) using the rules Joseph has build up. It seems like this would mean the Major gifts fund wouldn't make sense in the Fund option value set with this done

The other missing piece is adequately filling in the channel on ingress. This is something else we would work on.

Note that we further identified that Joseph calculates 'Donation Type' as the key field for Analytics. This doesn't exactly map to Account Code. I think we would both like there to be a value recorded on Ingress that is used both in the Financial Integration and the main analytics display & which is consistent with CiviCRM but I don't think we are thinking to do anything about that at this stage

Change #1192966 had a related patch set uploaded (by Eileen; author: Eileen):

[wikimedia/fundraising/crm@master] Add new field for tracking is_major_gift

https://gerrit.wikimedia.org/r/1192966

Change #1192966 merged by jenkins-bot:

[wikimedia/fundraising/crm@master] Add new field for tracking is_major_gift

https://gerrit.wikimedia.org/r/1192966

@Eileenmcnaughton Sorry that this is coming quite late in the game with the plan to add the field over the weekend. But I wonder if you've considered adding a Donation Type field (as you mentioned above) with a single option value of Major gift? Then if we get to the point of moving @JMando's donation type categorization algo to ingress, we won't need any further db changes. Of course that would only make sense if we're thinking that is something we want to do in the future.

If you do want to go down this road, I can take a look to review a patch on Sunday. But I imagine you've considered this and decided against it for the sake of simplicity.

@Lars good thinking.

On balance I think enough work needs to be done on the donation type field that there is no way it could be populated before the end of the year - in part because the donation_type field is not an exact map for any of the proposed finance fields, and that probably warrants a few conversations.

Adding a field / having a minor outage once we get into 2026 won't be a big deal - we do outages throughout the year to add fields / do minor upgrades but this last quarter is more of a problem

Thanks @Eileenmcnaughton, makes sense, just thought I'd check.

Moving this to done - I have separate phabs for finalising the 2 fields that are used to calculate the account code

XenoRyet set Final Story Points to 4.