How to Effectively Manage Duplicates Using Both HubSpot and Insycle's Detection Methods
Managing duplicate records in your database is an ongoing challenge that impacts data quality, reporting accuracy, and customer relationships. While HubSpot and Insycle both excel at finding duplicate records, they use different approaches to detection - which often leads to varying duplicate counts and can raise questions when trying to clean your data.
In this article, we'll explore the reasons these numbers differ and the best scenarios to utilize the unique features of each or leverage both systems together.
The discrepancy in duplicate numbers between HubSpot and Insycle is due to their fundamentally different approaches to identifying duplicates.
HubSpot's approach:
- Uses AI models for duplicate detection
- May consider a broader range of factors and patterns
- Results can be less predictable and may change over time as the AI model evolves
Insycle's approach:
- Uses a defined, deterministic method
- Based on specific rules and criteria set by the user
- Using templates, results are consistent and reproducible
Because of these different methodologies, the numbers will never be exactly the same between the two systems.
HubSpot's Manage duplicates feature is great when you can make quick decisions about obvious duplicates or want to maintain basic database hygiene.
HubSpot's AI-powered duplicate detection is ideal for:
- Getting a quick overview of potential duplicate issues in your database
- Handling straightforward merges where the matching criteria are obvious (like exact email matches)
- Keeping track of new duplicates that appear in your system through automated notifications
- Contact or company-focused duplicate clean-up
- Processing small batches of duplicates
Insycle shines when you need precise control over your deduplication process or when dealing with complex scenarios that require specific matching criteria.
Insycle's rule-based approach is best for:
- Finding duplicates based on specific field combinations or complex matching rules
- Processing large numbers of duplicate records at once
- Fuzzy matching; finding duplicates with slight variations in names, addresses, or other fields
- Reviewing and validating potential duplicates in bulk before merging
- Setting up scheduled deduplication processes with consistent rules
- Combining duplicate detection with other data-cleaning operations
You can make these differences work for you by using both systems together to create a more comprehensive duplicate detection strategy.
Use Duplicate Patterns to Build Smarter Merge Templates
Using HubSpot's AI model, you can better understand what field values the majority of duplicates in your CRM share. This info can offer direction on how to configure Insycle templates for managing the duplicates in bulk.
In Insycle’s Merge Duplicates module, under Step 4, configure rules to select the master record and which field values to copy into it.
On the Records tab, add rules for the fields that will help select the master record. Consider:
- What record has history that you want to preserve, and what property value can best identify this?
- What value is most important? Email address? Last activity? Most associations?
- What value is most unique?
On the Fields tab, define rules to pick which values from each duplicate group will be used in the master record. Consider:
- What values are scattered across different records that you want to consolidate into the master?
- What Criteria can be used to consistently identify the value you want to keep?
Align Your Insycle Matching Rules with HubSpot's Duplicate Detection
In these example results, the Insycle template was looking for similar matches in the First Name, Last Name, and Email fields.
Review the list of duplicates in HubSpot and identify duplicates that have not been found by Insycle. The contacts highlighted in the image below are a few examples of duplicates not found by the Insycle template.
Now, examine how HubSpot matched those records. Click the Review button next to a record, look at the properties, and note what is different about the values that match in those records.
In the additional matches HubSpot found, the emails are more varied than the ones Insycle picked up. In some cases, they don’t match at all. But all of the First Name and Last Name values match. Do they all have some other similar matching value? Here are some examples:
Contact Record | Name | Phone Number | Company Name | |
Ed Doe/ Edward Doe |
Similar name | Emails don't match | Same phone number | Companies don't match |
Mia Morgan | Same name | Emails don't match | Same phone number | Companies don't match |
Nuan Hsieh | Same name | Similar email | Both blank | Similar company |
Berat Saunders | Same name | Similar email | Phone doesn't match | Similar company |
Pepper Jones | Same name | Emails don't match | Similar phone number | One is blank |
Devon James | Same name | Emails don't match | Same phone number | Both blank |
In Insycle’s Merge Duplicates module, create new templates to catch these edge cases. Set up Step 1 to find duplicates based on the pattern you found in HubSpot. Based on the examples in the table above, you may want to create two new templates:
- Find similar matches in the First Name and Last Name fields,
and similar matches in the Phone Number field. - Find similar matches in the First Name and Last Name fields,
and similar matches in the Email and Company Name fields.
You may also need to adjust the Comparison Rules (Exact/Similar) or add Conditions such as “Empty allowed in any record.”
Now, when you click Find, do you see these additional matches?
Additional Resources
Related Help Articles
- Deduplicate HubSpot Contacts, Companies, and Deals in Bulk
- HubSpot Merge Duplicates Module Overview
- Deduplication Best Practices
Related Blog Posts