Bulk Merge Duplicate People, Companies

Navigate to Data Management > Merge Duplicates, pick the record type, and explore the default templates for a pre-built solution.

Step 1 looks through the records in your database, examining the fields that you specify for a match.

Click the + Field button to add fields you want to look at for duplicates, along with some parameters on what to look for. You want to choose fields that, in combination, give a high degree of certainty that the matched records are duplicates. See the Duplicate Identification Rules reference in the Module Overview for complete details.

For example, to find duplicate people, you may use the "First Name," "Last Name," and "Email Domain" fields. People with the same first name AND last name AND email domain will show as possible duplicates. 

When finished, click the Find button, and Insycle will generate a list of duplicates for you to review.

Expand Criteria for Matching Duplicates

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab. For example, you might want to look at both the Email and Additional Email fields for duplicate values.

The Conditions tab provides rules that one or more records in a duplicate group must meet. For complete details, see the Merge Duplicates Module Overview.

The CSV tab lets you upload a list of known duplicate record ID pairs directly into the Merge Duplicates module, bypassing field-based matching entirely. For complete details, see the CSV Tab Reference in the Module Overview. 

Records that have the same values in the fields specified in Step 1 are considered matches. When two or more records represent the same entity (person, company, or other) based on your matching rules, they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates based on your settings. For example, if you had four records for the same person, it would count as one duplicate group with four duplicate records.

Under Step 2, each row represents a duplicate group, with the number indicating the number of duplicate records found. When you click the chevron   at the right end of a duplicate group row to expand it and examine the individual records within that group. 

To exclude a duplicate group from all future deduplication analysis, click the X on the group row. Learn more about adding and removing duplicate groups from the Exclusions list in the Advanced How-Tos below.

Explore the record data in the duplicate groups. Double-check that the fields you set up in Step 1 are showing what you expected. 

Looking at the record details across several duplicate groups, decide the best way to determine which record is the master record that the other records will be merged into. Learn more about common practices for picking a master record.

Add more columns to the view using the gear icon on the right to get more context.

In Step 3, you define how your duplicates are merged. You'll choose your operation mode, set rules for which record becomes the master, and optionally control which field values are retained after the merge. 

Choose Your Operation Mode

At the top of Step 3, select Bulk or Manual mode:

• Bulk mode (recommended) — Automatically merges all duplicate groups according to your master selection rules. This is the right choice for most deduplication operations and is required for Templates, Recipes, and Automation.
• Manual mode — Lets you select specific records to merge one at a time from the Record Viewer. Reserve this for high-value records or complex situations that require hands-on review of each merge.

Most users should select Bulk mode and proceed.

Configure Rules to Automatically Select the Master Record (Master Tab)

On the Master tab, you define how all of the matching duplicate groups should be merged at scale. To do this, you need to create a series of rules that tell Insycle which record in each group should be selected as the master. The master is the record that will remain after the merge.

For example, if you had four records representing the same company, they would form a duplicate group of four records, which would be merged into a single master record. The other three records no longer exist.

For each duplicate group, Insycle will examine each Master rule to determine which records in the group meet the criteria. Rules are read in order, from top to bottom, and as soon as a record is the only one to meet a rule, it is selected as the master record. The subsequent rules are ignored. 

Select the processing method—By Priority or Absolute. Most de-duplication operations should use By Priority. 

Learn more about these options in the Master Selection Rules reference in the Module Overview article.

Setting Field Retention Rules (Fields Tab)

The Fields tab lets you control which field values end up in the master record after the merge. By default, Insycle keeps the master record's values and fills any empty fields with values from the most recently updated duplicate. You can override this field-by-field for any fields that need special handling.  

For example, you might want to retain the earliest created "Company Name" while keeping the "Phone Number" that appears most frequently.

For each field you want to control, select the field and specify where its data should be sourced. This is merged into the master. Insycle evaluates your rules from top to bottom and stops at the first match for each field. This means rule order matters—place your most specific rules first, with broader fallback rules below.

The Criteria dropdown gives you various options for choosing the data to keep, and the Group Fields lets you keep values of multiple fields from the same record.

Learn more about these options in the Field Data Retention Rules reference in the Module Overview article.

Select the Merge Method (HubSpot & Salesforce Only)

The Merge tab lets you choose between Native and Synthetic merge logic, and control whether empty master fields are automatically filled. See the Method Tab Reference for details.

Step 4 is where you preview and apply your merge changes.

Preview Merged Changes in CSV Report

Now with the filters and master record set up, you can preview the changes you are making to your data. That way, you can verify that your deduplication configuration is working as expected before those changes are pushed to your live database. 

Under Step 4, click the Review button and select Preview mode.

Click the Next button to go to the Notify screen, where you can select recipients for the email report. You can also add additional context on this screen.

On the When tab, click the Run Now tab, select which records to apply the change to (if you have many records, you might want to do only a few to test), and then click the Run Now button.

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application and review the data.

The CSV report includes records from all duplicate groups. It shows the current state of the master record, excluding reassignments of associated contacts, deals, or activities. (To validate how these reassociations occur, run a small-scale test in Update mode.) This ensures that the CSV accurately represents the merging setup before executing in Update mode.

There's a row for each record, which includes:

• The Result of the action
• A Message with details
• The Duplicate Group ID indicates which records will be merged together
• A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
• All fields used to identify the duplicates in Step 1
• The record ID, record name, or email, and Deeplink to the CRM record
• All fields selected for the Master and Fields rules in Step 3. Note that if a field is used on both tabs, it will only appear once in the CSV. 
• Any additional fields added to the Layout tab in Step 2.

 The Status column indicates:

• Duplicate – The record is part of a duplicate group.
• Master – The master record is chosen for the duplicate group based on default behavior and your Master rules. Review the selections in this row to determine whether the appropriate records are being chosen.
• Master (After) – For each duplicate group, the Master (After) row shows the values the final record will contain based on your Field rules and the default behavior.
• Error – If Insycle is not able to determine which record would be the master, an error message will appear here. See the Troubleshooting section below for more details.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your CRM Records

When you're satisfied with the preview results, you can apply the merge changes to your CRM.

Under Step 4, click the Review button again, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to the CRM. If you have a large number of records, consider doing a smaller batch to assess the results in your CRM.

If the Merge API is set to Synthetic on the Module tab in Step 3, clicking Run Now in Update Mode displays a Please confirm dialog before the operation begins. This dialog shows the number of records being merged and an estimated time for the operation.

Click Yes to proceed. 

Processing time for Synthetic merges depends on the volume of associated records and complexity of data to be reassigned; it will be longer than for Native merges — avoid running multiple operations in parallel on the same database. You can monitor progress in the Activity Tracker.

After you've seen the results in your CRM and are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a Recipe.

By automating with a template, you'll save time and ensure that your duplicates are merging consistently on an ongoing basis.

Save Your Configuration as a Template

Return to the Template menu at the top of the page and click Copy to save your configurations as a new version of whatever template you started with. Then click the pencil to edit your new template name.

Set Up Automation

Under Step 4, click the Review button, and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can also provide additional context in the message subject or body.

On the When tab, select Automate, and configure the frequency you'd like the template to run. When finished, click Schedule.

You can view all your scheduled automations at any time on the Operations > Automations page.

Bundle into a Recipe

When you have multiple templates that work reliably together — for example, separate templates for contacts, companies, and deals — you can combine them into a Recipe. A Recipe runs your templates in sequence, in the order you define, on a consistent schedule.

If your database is large or you're getting an overwhelming number of duplicate groups, use the Filter button in Step 1 to narrow down the records Insycle analyzes. Filtering to a subset — for example, contacts created in the last 30 days, or companies in a specific region — makes it easier to validate your configuration before running it across your full database. A filtered dataset also processes faster.

To add a filter, click the Filter button in Step 1, choose a field, select a condition, and set a value. The filter is applied before the matching step runs.

Most of the options in the Field dropdown match the fields that are found in your HubSpot records, and for contact records, there are three additional options related to the Email value: 

• Email Username: The portion of the email address before the “@.” For example, if the email address were “maria@acmewidgets.com,” the username value would be “maria.” 
• Free Email Provider Domain: Choose True to filter out records where the email domain is Gmail, Hotmail, Yahoo, or any of about 10,000 other free email providers. This filter helps ensure these are real clients, or can determine which record is the legitimate one, because it’s most likely that customer companies aren't using free Gmail accounts (though a contact may have accidentally emailed us from one at some point). 
• Email Top-Level Domain: The top-level domain (TLD) is everything that follows the final dot of a domain name. For example, in the domain name acmewidgets.com, '.com' is the TLD. Some other popular TLDs include '.org', '.uk', and '.edu'. 

When setting up your matching fields for the first time, start with your highest-confidence criteria — fields that, together, are very unlikely to match unless the records are true duplicates. Exact Match on email address plus first and last name is a reliable starting point for contacts. Domain Name is a strong starting point for companies.

Once you're confident your configuration is catching true duplicates without pulling in false positives, you can broaden your criteria — for example, by adding Similar Match to catch typos and slight variations: 

Or, by using Related Fields to compare values across two fields that contain similar data (such as Email and Additional Email).

Insycle evaluates your Master tab rules from Step 3 in order, eliminating records that don't match each rule until only one remains. If multiple records still match after all rules are evaluated, no master can be determined, and the group will show an error in the CSV.

For example, imagine having four duplicate records for the same contact. (In this image, we are examining the records in the Grid Edit module.)

In the Merge Duplicates module, you have configured the first three Master rules based on email engagement metrics, but all four records have identical values of zero, so no records are eliminated. Your fourth rule checks for an active Contact Owner — three records have no owner, so they're eliminated. The one remaining record becomes the master.

This is why rule order matters. Place your most reliably differentiating rules — like record owner, lifecycle stage, or engagement activity — where they're most likely to yield a clear winner. If you're frequently seeing errors in your CSV, revisit your rule order and consider adding a tiebreaker rule, such as earliest Create Date or latest Last Modified Date, as a final fallback.

In this CSV report example, you can see that the one record with an active owner was chosen as the master.

By default, all duplicate groups identified in Step 1 are candidates for merging. If there are specific groups you want to permanently prevent from being merged — regardless of which matching rules or templates you use — you can add them to the Duplicate Groups Exclusion List directly in Step 2.

To exclude a duplicate group:

In Step 2, click the X on a duplicate group row. The group will be added to the Exclusion List and will no longer appear in duplicate analysis or be included in merges, even when using different duplicate detection templates.

To review and manage excluded groups:

Click the Exclusions button in the Step 2 header. This opens a dialog showing all excluded groups. 

You can expand each group to view the individual records and their values, and use the Layout tab to configure which fields are displayed. To remove a group from the Exclusion List, click the X next to it — those records will once again be considered during duplicate analysis.

Important considerations:

• Exclusions apply to the entire duplicate group and cannot be applied to individual records within a group. If a group contains some records that are not actually duplicates, use Manual mode in Step 3 and uncheck the records that should not be merged.
• Exclusions are based on a specific set of record IDs. If a new duplicate record is later added to your CRM, the system may detect a new group containing that record alongside previously excluded records. Because the group now includes a different set of record IDs, it will appear again during duplicate analysis and can be evaluated or excluded as needed.

You have duplicate records identified by Insycle, but not all are merging into the master. Check how many duplicates are in the affected groups. If you have duplicate groups with more than 5 records, you may want to change the Exclusions value at the bottom of Step 3 on the Master tab: Skip duplicate groups with more than 5 records per group to ensure you can get them all.

This setting is intended to prevent overly broad matching criteria from accidentally merging unrelated records. You can adjust this threshold up to 100 records per group.

If the Result is "Failed" and the Message column of the CSV report displays this text:

Change rules in Step 3 'Master Selection'. Failed to pick master record because multiple records (X) meet the selection criteria. In 'Master Selection', change, add, or reorder the rules such that only one record matches (if cannot determine master based on field values, use 'Record ID is lowest' as the last rule).

This means Insycle could not determine which record in the duplicate group should become the master — none of the records meet more of the rules than the others.

There are a few things you can try to resolve this:

1. Under Step 3, on the Master tab, experiment with reordering or adding rules based on fields that are likely to have unique values across your records. Think about what matters most to your business — the fields that would make one record clearly more valuable or authoritative than another. For example:

   • A field that stores an ID linked to an external system, such as an ERP, data warehouse, or proprietary internal tool — the record with that ID populated is likely the one your other systems depend on
   • The record with the highest number of associated records (contacts, deals, activities) — indicating it has accumulated more history and relationships
   • A field indicating CRM ownership or account status — for example, a record marked as "Customer" vs. "Lead," or one with an active owner assigned
   • A field that tracks revenue, subscription tier, or engagement level — the record with higher values may represent the more complete or current entry
   • A custom field your team uses to flag record quality or data source reliability

   The goal is to surface the fields that reflect record importance for your specific operations, not just default system fields.

2. Also on the Step 3 Master tab, confirm that By Priority is selected rather than Absolute.

   With By Priority, a record only needs to match one rule to be selected as master. With Absolute, a record must meet all rule criteria, which makes it much harder for any single record to qualify.

   In most cases, By Priority is the right choice. If By Priority is already selected and the error persists, none of the records in the group meet any of the criteria more than the others. Return to the Master tab and continue adding or reordering rules, focusing on the business-critical fields described in #1 above.

   

3. As a last resort, add a rule on the Master tab that uses Record ID is lowest, or Create Date is earliest. These rules will always produce a unique winner, ensuring master selection can complete even when other field-based rules cannot differentiate the records.

There are a couple of things to look at that may be causing records to be misidentified as duplicates.

First, you may need a better unique identifier. Under Step 1, if you use only fields that can correctly contain the same values across multiple records, they aren't unique identifiers. In this case, you are likely to identify unrelated records as duplicates and may accidentally merge them.

Unique identifiers are data unlikely to be shared by any other record unless they represent the same underlying entity. Common fields used for deduplication include phone numbers, email addresses, mailing addresses, and ID numbers.

Second, this may indicate the Comparison Rule under Step 1 is too broad. Try using the Exact Match comparison rule instead of the Similar Match rule. Similar Match looks for values that may be close, even with a one-character difference (e.g., a typo), broadening the search. 

Remember to always run your deduplication in Preview Mode to confirm it's working as expected before running it in Update Mode and applying the changes to your CRM records.

Field rules in Step 3 can only update writable fields. Read-only fields are not shown in the Fields tab dropdown because the CRM prevents these properties from being written back. Insycle excludes them from the field selection options to prevent errors.

While you cannot create field rules for read-only properties in the Fields tab, you can use them in the Master tab. Read-only fields, such as "Last Activity Date," can be used to make merge decisions without attempting to modify those properties.

You can also use read-only properties for filtering and matching in Step 1. Read-only fields are also available when reviewing data in Step 2 or in the CSV report.

To identify which fields are read-only, use the Cleanse Data module. Navigate to Data Management > Cleanse Data, then select a database and record type from the top menu. You can search for a specific field by name or click the Writable heading in the table to sort and review all writable or non-writable fields. A checked box in the Writable column indicates the field is writable.

It can take Insycle a while to find and match duplicates when the fields used to identify them have very long values. The longer the values, the longer it takes Insycle to process the data and generate the results. This might come up when looking for matches based on long ID numbers, LinkedIn bio links, or other URLs with long strings attached (ex, https://www.linkedin.com/in/svadin%C3%ADr-n%C4%9Bmec-1234b31a3/).

You can speed this up by limiting how much of the value Insycle looks at.

If the beginning or ending portion of the values are all unique, you can limit the comparison to the first or last several characters using the Match Parts parameter under Step 1. 

Or use the Ignore Text (Substrings) parameter, then click the Terms button.

On the Ignored Text tab of the popup, add the common portion of the URL or text string.

Field rules in Step 3 can only update writable fields. Read-only fields are not shown in the Fields tab dropdown because the CRM prevents these properties from being written back. Insycle excludes them from the field selection options to prevent errors.

While you cannot create field rules for read-only properties in the Fields tab, you can use them in the Master tab. Read-only fields, such as "Last Activity Date," can be used to make merge decisions without attempting to modify those properties.

You can also use read-only properties for filtering and matching in Step 1. Read-only fields are also available when reviewing data in Step 2 or in the CSV report.

To identify which fields are read-only, use the Cleanse Data module. Navigate to Data Management > Cleanse Data, then select a database and record type from the top menu. You can search for a specific field by name or click the Writable heading in the table to sort and review all writable or non-writable fields. A checked box in the Writable column indicates the field is writable.

When you initiate a merge operation in Insycle, you will see the estimated time to complete. If you notice that the operation is taking longer than expected, there are several factors to consider:

• If you are using a synthetic merge, this will take longer to run than a native merge.
• Complex processes, like merges and associations, can take a while. This is because each backend task—from updating associations to changing properties—contributes to the total processing time. For instance, when merging three deals into one, 20-40 operations may be required to update and reparent all related activities from the duplicates into the master.
• Throughput is measured across all operations; when multiple operations run in parallel, they increase processing load and slow each other down. To optimize processing time, run only one operation at a time.

There are two main ways to monitor the progress of a long-running operation:

Option One: Periodically check the Activity Tracker

The operation will appear in the Activity Tracker once it is complete.

Option Two: Periodically refresh the template for the operation

Open the template for the operation. Under Step 1, click the Find button.  As the merge continues, you should see the number of duplicate records gradually decrease in the Record Viewer, which indicates the process is actively working through them.

Look at the number of results in the Record Viewer.

Wait a few minutes and refresh the page. You should see the number of results decrease as Insycle processes the operation.

Currently, neither HubSpot nor Salesforce provides an automated way to prioritize active owners during the merge process. You'd need to verify owner status manually for each merge operation.

However, Insycle's Merge Duplicates module includes an option to prioritize an active owner. 

First, you could add a Master rule under Step 3 to tell Insycle to select the record from each group with an active owner as the master record.

Add a rule with the following parameters:

• Field: Record owner
• Condition: active user

Second, you could create a Field rule to retain the owner who is an active user.

Add a rule with the following parameters:

• Field: Owner
• Criteria: From record where value
• Condition: active user

There are two ways to ensure the records you are merging are indeed duplicates.

First, always run your deduplication templates in Preview Mode before running them in Update Mode. This produces a CSV file showing how your records would have been merged. Then you can ensure that your Merge Duplicates template is working as expected and not merging non-duplicate records together.

Additionally, to ensure a smooth merge, consider narrowing the matching settings in Step 1. Try the Exact Match Comparison Rule instead of Similar Match. Then make sure that you are using actual, uniquely identifying fields—first name, last name, email, and phone number are popular choices. The more tightly defined your filter is, the less likely you are to merge non-duplicate records.

If the Message column of the CSV report displays this error:

Change rules in Step 3 'Master Selection'. Failed to pick master record because multiple records (X) meet the selection criteria. In 'Master Selection', change, add, or reorder the rules such that only one record matches (if cannot determine master based on field values, use 'Record ID is lowest' as the last rule).

None of the records meet more of the rules than the others do.

There are a couple of things you can try to resolve this:

1. On the Master tab in Step 3, experiment with reordering or adding additional fields that are likely to have unique values.
2. At the bottom of the Master tab in Step 3, ensure By Priority is selected, not Absolute.
   With By Priority, your master record only has to match one rule. Using Absolute, your master record would have to meet all of the rule criteria. In most cases, it is best to select By Priority.
   If By Priority was used, then none of the records in the duplicate group meet any of the criteria on the list more than the others. In this case, you'll need to experiment, reordering or adding additional rules for fields likely to have unique values.
3. As a last resort, you can add a rule on the Master tab in Step 3 that says Record ID is lowest, or Create Date is earliest.

Yes, Insycle allows you to select which field data to retain in the master record using the Fields tab in Step 3. See the Merge Duplicates module field data retention rules reference for more details.

Yes, there are several ways to share details and get approval before merging duplicates.

You can manually approve master records and mark them in a CSV file, then use Insycle to bulk deduplicate into those master records. See the Customize Bulk Deduplication Using Exclusions and Pre-Defined Masters article to learn more.

Or, you can run the Merge Duplicates module in Preview Mode and then deliver the preview CSV that Insycle generates. The CSV report includes your entire merge operation down to individual duplicate groups, but does not update your live data. Then your team can approve the merge based on this report, before running Merge Duplicates in Update Mode.

Additionally, team members can review duplicates and manually select the master for each record under Step 3 by selecting Manual mode. Review the Manually Merge Duplicates article for more details.

No, matching fields do not need to match exactly. The Similar Match Comparison Rule in Step 1 acts as a fuzzy match, looking for values that may be close but differ by only one character (e.g., a typo), and broadens the search.

This search behaves like when Google shows results for a slightly different term or says, “Did you mean...” For example, if an Email of “huey@coahulldu.co” is found, it could include records with the values “hueyy@coahulldu.co" or "hue.y@coahulldu.co” as a match.

You should be careful when using Similar Match as the looser criteria can incorrectly identify non-duplicates as duplicates. 

Review the Understanding Similar Matching best practices for more details.

By default, merged records retain the value from the master record. If you want to keep a specific value from a read-only field that's not in the master, you have two options:

1. Copy to a writable custom property: In your CRM, create a custom field to copy the read-only field value into. Then run a Copy operation in the Transform Data module before performing the merge. Finally, set up Merge Duplicates rules on the Fields tab of Step 3 to retain the value of that custom property that meets your criteria.

2. Use master selection rules: Set the master selection rules on the Master tab of Step 3 to retain the record with the read-only value you want to keep. This guarantees the correct record becomes the master without trying to change the read-only property itself.

Yes. You can use an existing list of duplicates with Insycle to deduplicate it in bulk, following these steps:

1. Prepare a CSV file with columns for the record IDs and a "Merge Master" column. In the "Merge Master" column, mark which record should be kept after merging.
2. Create a custom field called "Merge Master" in your CRM.
3. Use the Magical Import module to import your CSV file into the CRM, updating the "Merge Master" field for the relevant records.
4. Go to the Merge Duplicates module and set up a filter to select records based on the "Merge Master" field.

Learn more about deduplicate records using a CSV.

Insycle displays 50 records on the module screen as a preview; this is not the full list. Insycle can process thousands of duplicate groups in a single operation. You could potentially deduplicate your entire database in a single step. 

When using the Preview in Step 4, select the "All" option under When > Run Now to view all records in the Preview CSV report. When you run the operation in Update mode, all records will automatically be included.

You can merge up to 100 duplicate records into a single master record.

By default, Insycle skips any duplicate group containing more than 5 records. If you have duplicate groups with more than 5 records, you may want to adjust the Exclusions setting at the bottom of the Master tab in Step 3. The Skip duplicate groups with more than X records per group setting can be increased up to 100 records per group to ensure all duplicates in large groups are processed.

This default limit is a precaution to prevent duplicate matching criteria that are too broad in Step 1 from accidentally merging many non-duplicate records. If you plan to set this number higher, it is a good idea to run Preview Mode first to ensure your deduplication template is operating as you intend.

All plans include unlimited usage, unlimited users, and unlimited operations. During the free trial, there is a cap of 500 records updated, cleansed, or merged. See the pricing page for more details.