Page MenuHomePhabricator

Find source of unlimited dedupe queries, prevent them
Closed, ResolvedPublic2 Estimated Story Points

Description

A Civi user was doing normal email searches and merges from the search results, but seems to have kicked off an unlimited scan of the email table for duplicates, locking it for long enough to block a couple dozen contribution inserts. She says she never saw the dedupe search screen (where we enforce a limit)

Instrument the dedupe queries to log a full stack trace, and enforce a limit at a level lower than the UI

Event Timeline

Ejegg triaged this task as High priority.Sep 8 2017, 5:22 PM
Ejegg set the point value for this task to 2.

Change 376760 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/crm@master] Log a stack trace for all dedupe queries

https://gerrit.wikimedia.org/r/376760

Ejegg moved this task from Backlog to Review on the Fundraising Sprint Quill Pencil board.

Patch for review is just the diagnostic part of this ticket.

Change 376760 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Log a stack trace for all dedupe queries

https://gerrit.wikimedia.org/r/376760

Change 379601 had a related patch set uploaded (by Ejegg; owner: Ejegg):
[wikimedia/fundraising/crm@master] WIP Use permissions hook to prevent unlimited dedupe

https://gerrit.wikimedia.org/r/379601

Change 379601 merged by jenkins-bot:
[wikimedia/fundraising/crm@master] Use permissions hook to prevent unlimited dedupe

https://gerrit.wikimedia.org/r/379601

We've deployed a couple of things to block the bad queries, but we're still not sure exactly where the bad redirect is coming from in Civi core

Go for it, and best of luck!

Change 381311 had a related patch set uploaded (by Eileen; owner: Eileen):
[wikimedia/fundraising/crm/civicrm@master] CRM-21224 reinstate use of limit on dedupe searches

https://gerrit.wikimedia.org/r/381311

Change 381311 merged by jenkins-bot:
[wikimedia/fundraising/crm/civicrm@master] CRM-21224 reinstate use of limit on dedupe searches

https://gerrit.wikimedia.org/r/381311

i think we got this .... unless it come back in some way