Page MenuHomePhabricator
Create Task
Maniphest T236977

Audit & support for benevity import
Closed, ResolvedPublic
Actions

Description

I renamed this to what it has become - will open another for the slowness

Details

SubjectRepoBranchLines +/-
Fix issue where source field was sometimes messing with importswikimedia/fundraising/crmmaster+9 -5
Add message support for recovering from a borked importwikimedia/fundraising/crmmaster+1 -1
Exclude major matching gifts donors from totals calcwikimedia/fundraising/crmmaster+15 -3
Customize query in gerrit

Event Timeline

RLewis created this task.Oct 31 2019, 4:35 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 31 2019, 4:35 AM
Aklapper added projects: Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM.Oct 31 2019, 8:55 AM
Aklapper removed subscribers: Fundraising-Backlog, Wikimedia-Fundraising-CiviCRM.
• DStrine moved this task from Triage to Sprint +1 on the Fundraising-Backlog board.Oct 31 2019, 4:42 PM
• DStrine added a project: Fundraising Sprint Visual Basic Instinct.Nov 4 2019, 10:14 PM
• DStrine moved this task from Sprint +1 to Current Sprint on the Fundraising-Backlog board.
Eileenmcnaughton added a comment.Nov 5 2019, 12:40 AM

Opps this wasn't the one to move in - we should leave this until after BE - @RLewis - what was the one that was about auditing the totals?

RLewis added a comment.Nov 5 2019, 5:08 AM

@Eileenmcnaughton just adding in another issue i've encountered when importing a benevity file for the EUR currency file.

I've added the file to the server under the Benevity folder, October 2019 - error eur file

Eileenmcnaughton renamed this task from Benevity import taking a long time with large files to Audit & support for benevity import.Nov 12 2019, 9:15 PM
Eileenmcnaughton updated the task description. (Show Details)
• DStrine added a project: Fundraising Sprint A Wrinkle in Timezones.Nov 12 2019, 9:28 PM
• DStrine added a project: Fundraising Sprint X-rays.Nov 26 2019, 8:56 PM
Eileenmcnaughton moved this task from Backlog to Doing on the Fundraising Sprint X-rays board.Nov 27 2019, 4:57 AM
• DStrine added a project: Fundraising Sprint YAMLton, the Musical.Dec 10 2019, 9:17 PM
Eileenmcnaughton moved this task from Backlog to Doing on the Fundraising Sprint YAMLton, the Musical board.Dec 11 2019, 12:27 AM
gerritbot added a comment.Dec 12 2019, 7:29 PM

Change 556775 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Exclude major matching gifts donors from totals calc

https://gerrit.wikimedia.org/r/556775

gerritbot added a project: Patch-For-Review.Dec 12 2019, 7:29 PM
gerritbot added a comment.Dec 12 2019, 8:01 PM

Change 556775 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Exclude major matching gifts donors from totals calc

https://gerrit.wikimedia.org/r/556775

Maintenance_bot removed a project: Patch-For-Review.Dec 12 2019, 8:10 PM
gerritbot added a comment.Dec 13 2019, 4:56 AM

Change 556860 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Add message support for recovering from a borked import

https://gerrit.wikimedia.org/r/556860

gerritbot added a project: Patch-For-Review.Dec 13 2019, 4:57 AM
gerritbot added a comment.Dec 13 2019, 5:10 AM

Change 556861 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm@master] Fix issue where source field was sometimes messing with imports

https://gerrit.wikimedia.org/r/556861

Eileenmcnaughton added a subscriber: NNichols.Dec 13 2019, 8:37 AM

after some attempts I determined that the speed issue with matching gift donors with many gifts was getting worse the more contributions we added at a rapid rate. I’m pretty sure this indicates the join for the wmf_donor fields just won’t scale to 20k+ donations on a single donor. I worked with Elliott & Dallas to alter the triggers to exclude our most significant matching gift donors. I discussed with @NNichols & if not having that info for those 2 contacts ever becomes an issue she will raise a phab then. I had thought that we would look at doing this in Jan but it became apparently it would be impossible to get the file in if we left it until then & we had opportunity with queues relatively slow so we did it.

Eileenmcnaughton moved this task from Doing to Deployed on the Fundraising Sprint YAMLton, the Musical board.Dec 13 2019, 8:38 AM
gerritbot added a comment.Dec 17 2019, 1:36 AM

Change 556860 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Add message support for recovering from a borked import

https://gerrit.wikimedia.org/r/556860

gerritbot added a comment.Dec 17 2019, 6:54 PM

Change 556861 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Fix issue where source field was sometimes messing with imports

https://gerrit.wikimedia.org/r/556861

Maintenance_bot removed a project: Patch-For-Review.Dec 17 2019, 7:10 PM
Eileenmcnaughton added a comment.Dec 22 2019, 10:20 PM

I added this upstream https://github.com/civicrm/civicrm-core/pull/16143 for the confusing financial_type_id is not valid message - I don't think we need to backport it now we know it's a deadlock

• DStrine closed this task as Resolved.Jan 7 2020, 9:05 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL