Bulk Merge Duplicate People, Companies

Navigate to Data Management > Merge Duplicates, pick the record type, and explore the default templates for a pre-built solution.

Step 1 looks through the records in your database, examining the fields that you specify for a match.

Click the + Field button to add fields you want to look at for duplicates, along with some parameters on what to look for. You want to choose fields that, in combination, give a high degree of certainty that the matched records are duplicates. See the Setting the Criteria for Finding Duplicate Matches article for complete details.

For example, to find duplicate people, you may use the "First Name," "Last Name," and "Email Domain" fields. People with the same first name AND last name AND email domain will show as possible duplicates. 

When finished, click the Find button, and Insycle will generate a list of duplicates for you to review.

Expand Criteria for Matching Duplicates

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab. For example, you might want to look at both the Email and Additional Email fields for duplicate values.

The Conditions tab provides rules that one or more records in a duplicate group must meet.

The CSV tab lets you upload a list of known duplicate record ID pairs directly into the Merge Duplicates module, bypassing field-based matching entirely. For complete details, see the Merging Duplicates from a List You Already Have article. 

Records that have the same values in the fields specified in Step 1 are considered matches. When two or more records represent the same entity (person, company, or other) based on your matching rules, they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates based on your settings. For example, if you had four records for the same person, it would count as one duplicate group with four duplicate records.

Under Step 2, each row represents a duplicate group, with the number indicating the number of duplicate records found. When you click the chevron   at the right end of a duplicate group row to expand it and examine the individual records within that group. 

To exclude a duplicate group from all future deduplication analysis, click the X on the group row. Learn more about adding and removing duplicate groups from the Exclusions list in the Advanced How-Tos below.

Explore the record data in the duplicate groups. Double-check that the fields you set up in Step 1 are showing what you expected. 

Looking at the record details across several duplicate groups, decide the best way to determine which record is the master record that the other records will be merged into. Learn more about common practices for picking a master record.

Add more columns to the view using the gear icon on the right to get more context.

In Step 3, you define how your duplicates are merged. You'll choose your operation mode, set rules for which record becomes the master, and optionally control which field values are retained after the merge. 

Choose Your Operation Mode

At the top of Step 3, select Bulk or Manual mode:

• Bulk mode (recommended) — Automatically merges all duplicate groups according to your master selection rules. This is the right choice for most deduplication operations and is required for Templates, Recipes, and Automation.
• Manual mode — Lets you select specific records to merge one at a time from the Record Viewer. Reserve this for high-value records or complex situations that require hands-on review of each merge.

Most users should select Bulk mode and proceed.

Configure Rules to Automatically Select the Master Record (Master Tab)

On the Master tab, you define how all of the matching duplicate groups should be merged at scale. To do this, you need to create a series of rules that tell Insycle which record in each group should be selected as the master. The master is the record that will remain after the merge.

For example, if you had four records representing the same company, they would form a duplicate group of four records, which would be merged into a single master record. The other three records no longer exist.

For each duplicate group, Insycle will examine each Master rule to determine which records in the group meet the criteria. Rules are read in order, from top to bottom, and as soon as a record is the only one to meet a rule, it is selected as the master record. The subsequent rules are ignored. 

Select the processing method—By Priority or Absolute. Most de-duplication operations should use By Priority. 

Learn more about these options in the Choosing Which Record Remains After a Merge article.

Setting Field Retention Rules (Fields Tab)

The Fields tab lets you control which field values end up in the master record after the merge. By default, Insycle keeps the master record's values and fills any empty fields with values from the most recently updated duplicate. You can override this field-by-field for any fields that need special handling.  

For example, you might want to retain the earliest created "Company Name" while keeping the "Phone Number" that appears most frequently.

For each field you want to control, select the field and specify where its data should be sourced. This is merged into the master. Insycle evaluates your rules from top to bottom and stops at the first match for each field. This means rule order matters—place your most specific rules first, with broader fallback rules below.

The Criteria dropdown gives you various options for choosing the data to keep, and the Group Fields lets you keep values of multiple fields from the same record.

Learn more about these options in the Keeping the Right Data When Records Merge article.

Select the Merge Method (HubSpot & Salesforce Only)

The Merge tab lets you choose between Native and Synthetic merge logic, and control whether empty master fields are automatically filled. See Native vs. Custom Merge: Two Ways to Combine Records in HubSpot and Salesforce for details.

Step 4 is where you preview and apply your merge changes.

Preview Merge Changes in CSV Report

Now with the filters and master record set up, you can preview the changes you are making to your data. That way, you can verify that your deduplication configuration is working as expected before those changes are pushed to your live database. 

Under Step 4, click the Review button and select Preview mode.

Click the Next button to go to the Notify screen, where you can select recipients for the email report. You can also add additional context on this screen.

On the When tab, click the Run Now tab, select which records to apply the change to (if you have many records, you might want to do only a few to test), and then click the Run Now button.

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application and review the data.

The CSV report includes records from all duplicate groups. It shows the current state of the master record, excluding reassignments of associated contacts, deals, or activities. (To validate how these reassociations occur, run a small-scale test in Update mode.) This ensures that the CSV accurately represents the merging setup before executing in Update mode.

There's a row for each record, which includes:

• The Result of the action
• A Message with details
• The Duplicate Group ID indicates which records will be merged together
• A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
• All fields used to identify the duplicates in Step 1
• The record ID, record name, or email, and Deeplink to the CRM record
• All fields selected for the Master and Fields rules in Step 3. Note that if a field is used on both tabs, it will only appear once in the CSV. 
• Any additional fields added to the Layout tab in Step 2.

 The Status column indicates:

• Duplicate – The record is part of a duplicate group.
• Master – The master record is chosen for the duplicate group based on default behavior and your Master rules. Review the selections in this row to determine whether the appropriate records are being chosen.
• Master (After) – For each duplicate group, the Master (After) row shows the values the final record will contain based on your Field rules and the default behavior.
• Error – If Insycle is not able to determine which record would be the master, an error message will appear here. See the Merge Duplicates Troubleshooting article for more details.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your CRM Records

When you're satisfied with the preview results, you can apply the merge changes to your CRM.

Under Step 4, click the Review button again, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to the CRM. If you have a large number of records, consider doing a smaller batch to assess the results in your CRM.

If the Merge API is set to Synthetic on the Module tab in Step 3, clicking Run Now in Update Mode displays a Please confirm dialog before the operation begins. This dialog shows the number of records being merged and an estimated time for the operation.

Click Yes to proceed. 

Processing time for Synthetic merges depends on the volume of associated records and complexity of data to be reassigned; it will be longer than for Native merges — avoid running multiple operations in parallel on the same database. You can monitor progress in the Activity Tracker.

After you've seen the results in your CRM and are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a Recipe.

By automating with a template, you'll save time and ensure that your duplicates are merging consistently on an ongoing basis.

Save Your Configuration as a Template

Return to the Template menu at the top of the page and click Copy to save your configurations as a new version of whatever template you started with. Then click the pencil to edit your new template name.

Set Up Automation

Under Step 4, click the Review button, and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can also provide additional context in the message subject or body.

On the When tab, select Automate, and configure the frequency you'd like the template to run. When finished, click Schedule.

You can view all your scheduled automations at any time on the Operations > Automations page.

Bundle into a Recipe

When you have multiple templates that work reliably together — for example, separate templates for contacts, companies, and deals — you can combine them into a Recipe. A Recipe runs your templates in sequence, in the order you define, on a consistent schedule.

If your database is large or you're getting an overwhelming number of duplicate groups, use the Filter button in Step 1 to narrow down the records Insycle analyzes. Filtering to a subset — for example, contacts created in the last 30 days, or companies in a specific region — makes it easier to validate your configuration before running it across your full database. A filtered dataset also processes faster.

To add a filter, click the Filter button in Step 1, choose a field, select a condition, and set a value. The filter is applied before the matching step runs.

Most of the options in the Field dropdown match the fields that are found in your HubSpot records, and for contact records, there are three additional options related to the Email value: 

• Email Username: The portion of the email address before the “@.” For example, if the email address were “maria@acmewidgets.com,” the username value would be “maria.” 
• Free Email Provider Domain: Choose True to filter out records where the email domain is Gmail, Hotmail, Yahoo, or any of about 10,000 other free email providers. This filter helps ensure these are real clients, or can determine which record is the legitimate one, because it’s most likely that customer companies aren't using free Gmail accounts (though a contact may have accidentally emailed us from one at some point). 
• Email Top-Level Domain: The top-level domain (TLD) is everything that follows the final dot of a domain name. For example, in the domain name acmewidgets.com, '.com' is the TLD. Some other popular TLDs include '.org', '.uk', and '.edu'. 

When setting up your matching fields for the first time, start with your highest-confidence criteria — fields that, together, are very unlikely to match unless the records are true duplicates. Exact Match on email address plus first and last name is a reliable starting point for contacts. Domain Name is a strong starting point for companies.

Once you're confident your configuration is catching true duplicates without pulling in false positives, you can broaden your criteria — for example, by adding Similar Match to catch typos and slight variations: 

Or, by using Related Fields to compare values across two fields that contain similar data (such as Email and Additional Email).

Insycle evaluates your Master tab rules from Step 3 in order, eliminating records that don't match each rule until only one remains. If multiple records still match after all rules are evaluated, no master can be determined, and the group will show an error in the CSV.

For example, imagine having four duplicate records for the same contact. (In this image, we are examining the records in the Grid Edit module.)

In the Merge Duplicates module, you have configured the first three Master rules based on email engagement metrics, but all four records have identical values of zero, so no records are eliminated. Your fourth rule checks for an active Contact Owner — three records have no owner, so they're eliminated. The one remaining record becomes the master.

This is why rule order matters. Place your most reliably differentiating rules — like record owner, lifecycle stage, or engagement activity — where they're most likely to yield a clear winner. If you're frequently seeing errors in your CSV, revisit your rule order and consider adding a tiebreaker rule, such as earliest Create Date or latest Last Modified Date, as a final fallback.

In this CSV report example, you can see that the one record with an active owner was chosen as the master.

Even though this was the fifth rule, it is the first one that matched only one record:

• When Insycle processed the first three rules—Marketing emails clicked, emails bounced, and emails opened—Insycle couldn't eliminate any records because they all had a value of 0.

• None of the record email addresses are "role-based," so there isn't a single choice for the master using the Email rule either.

• In the next rule about contact owner, there is one record with an active contact owner; therefore, that is the master record.

By default, all duplicate groups identified in Step 1 are candidates for merging. If there are specific groups you want to permanently prevent from being merged — regardless of which matching rules or templates you use — you can add them to the Duplicate Groups Exclusion List directly in Step 2.

 Note that this feature applies only to Bulk mode. If you are using Manual mode in Step 3, use the record checkboxes to deselect individual records instead.

To exclude a duplicate group:

In Step 2, click the X on a duplicate group row. The group will be added to the Exclusion List and will no longer appear in duplicate analysis or be included in merges, even when using different duplicate detection templates.

To review and manage excluded groups:

Click the Exclusions button in the Step 2 header. This opens a dialog showing all excluded groups. 

You can expand each group to view the individual records and their values, and use the Layout tab to configure which fields are displayed. To remove a group from the Exclusion List, click the X next to it — those records will once again be considered during duplicate analysis.

Important considerations:

• Exclusions apply to Bulk mode only and to the entire duplicate group — they cannot be applied to individual records within a group. If you need to exclude a single record rather than the entire group, switch to Manual mode in Step 3 and uncheck the records that should not be merged.
• Exclusions are based on a specific set of record IDs. If a new duplicate record is later added to your CRM, the system may detect a new group containing that record alongside previously excluded records. Because the group now includes a different set of record IDs, it will appear again during duplicate analysis and can be evaluated or excluded as needed.