Page MenuHomePhabricator
Create Task
Maniphest T363965

Fail mail on dedupe
Open, MediumPublic
Actions

Description

In discussion with @AKanji-WMF & @Ejegg today we decided to get the dedupe script to re-run though the entire database as it is able to clean up a lot of duplicates that it missed on previous runs as our conflict handling has improved over time. I'm seeing between 1% & 20% on blocks of contacts (I'm sure it varies a lot depending on which blocks are being tackled).

However, after enabling we started to get fail mail. These fail mails occur when specific contact pairs are attempted. As an interim fix @Ejegg has slowed down the rate of this job so it doesn't annoy us too much until we resolve it.

The error is a type hint & would be easily solved by changing / handling that although ideally we would capture the scenario in a unit test & prevent any recurrence

I can reliably replicate the error on staging with this pair

drush @wmff cvapi Contact.merge to_keep_id=135090 to_remove_id=299459 mode=safe debug=1

TypeError: Argument 2 passed to [error]
CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver::isReHomingRequired() must be of the type array, null
given, called in
/srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php
on line 126 in CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver->isReHomingRequired() (line 42 of
/srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php).
Cannot modify header information - headers already sent by (output started at [warning]
/srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php:122)
bootstrap.inc:1551
TypeError: Argument 2 passed to CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver::isReHomingRequired() must be of the type array, null given, called in /srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php on line 126 in CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver->isReHomingRequired() (line 42 of /srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php).

Related Objects

Event Timeline

Eileenmcnaughton created this task.Thu, May 2, 5:47 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptThu, May 2, 5:47 AM
AKanji-WMF triaged this task as Medium priority.Thu, May 2, 4:56 PM
AKanji-WMF moved this task from Triage to DRI Backlog on the Fundraising-Backlog board.
Ejegg added a comment.EditedThu, May 2, 7:21 PM

There's another class of failmail coming in, e.g. contacts 730314 and 180366:

Undefined offset: 2 MergeHandler.php:891
Undefined offset: 2 MergeHandler.php:287
TypeError: Return value of CRM_Deduper_BAO_MergeHandler::getAddressBlock() must be of the type array, null returned in CRM_Deduper_BAO_MergeHandler->getAddressBlock() (line 287 of sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/MergeHandler.php).
CRM_Deduper_BAO_MergeHandler::getAddressBlock() must be of the type array, null returned in ...

One contact has 3 addresses, the other has 2. It might have to do with the order of the address types

The 2 addresses for record 1 are
0 => [ location_type_id => 1 ]
1 => [ location_type_id => 11 ]

The 3 addresses for record 2 are
0 => [ location_type_id => 4 ]
1 => [ location_type_id => 1 ]
2 => [ location_type_id => 11 ]

The addresses with location type 11 are different, so that would be the conflict. MergeHandler seems to expect they will both be at index 2 in the array at
$this->dedupeData['migration_info']['main_details']['location_blocks']['address']

jgleeson added a project: Fundraising Tech - Chaos Crew.Fri, May 3, 4:10 PM
AKanji-WMF added a comment.Mon, May 6, 8:29 PM

adding in from: T363962: Dedupe failure on typing

2024-05-01 23:42:02,917	INFO	TypeError: Argument 2 passed to CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver::isReHomingRequired() must be of the type array, null given, called in /srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php on line 123 in CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver->isReHomingRequired() (line 42 of /srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php).
2024-05-01 23:42:02,917	ERROR	CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver::isReHomingRequired()
2024-05-01 23:42:02,918	ERROR	must be of the type array, null given, called in
2024-05-01 23:42:02,918	ERROR	/srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php
2024-05-01 23:42:02,918	ERROR	on line 123 in
2024-05-01 23:42:02,918	ERROR	CRM_Deduper_BAO_Resolver_PreferredContactLocationResolver->isReHomingRequired()
2024-05-01 23:42:02,918	ERROR	(line 42 of
2024-05-01 23:42:02,918	ERROR	/srv/org.wikimedia.civicrm/drupal/sites/default/civicrm/extensions/deduper/CRM/Deduper/BAO/Resolver/PreferredContactLocationResolver.php).
2024-05-01 23:42:02,918	ERROR	Drush command terminated abnormally due to an unrecoverable error.
AKanji-WMF mentioned this in T363962: Dedupe failure on typing.Mon, May 6, 8:29 PM
AKanji-WMF moved this task from Backlog to Chaos on the FR-Civi-Dedupe board.Wed, May 15, 8:06 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL