Weird duplicates cartoon

How to Analyze and Merge Duplicates Manually for Granular Deduplication Control

You have duplicate records in your CRM, but you need a controlled, careful process to merge these records, so bulk merging isn't an option.

With the Merge Duplicates module, you can surface duplicates, analyze the records to determine which have the relevant data, and merge them manually, one at a time. 

Process Summary

  1. Identify duplicates.
  2. Review and analyze the duplicates.
  3. Select the records that need merging.
  4. Choose the master record.
  5. Individually select or modify values to be retained.
  6. Merge duplicate records.

 

Step-by-Step Instructions

Configure Rules to Identify Duplicates

Navigate to the Merge Duplicates module, pick the object type, and explore the default templates for a pre-built solution.

To find duplicates, you need to define how to match records. Step 1 looks through the records in your database, examining the fields that you specify for matches. Each row is for a field you want to look at for duplicates. 

For example, to find duplicate Contacts, you may use the "First Name," "Last Name," and "Email Domain" fields. Contacts with the same first name AND last name AND email domain will show as possible duplicates. 

Choose fields that, in combination, give a high degree of certainty that the matched records are duplicate records. 

merge-duplicates-hubspot-contacts-step-1-first-last-email-domain-646w.png

See the Merge Duplicates Module Overview for complete details on selecting fields to use and narrowing your results with the filter.

When finished, click the Find button, and Insycle will generate a list of duplicates for you to review.

Expand Criteria for Matching Duplicates

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab. For example, you might want to look at both the Email and Additional Email fields for duplicate values.

merge-duplicates-hubspot-contacts-step-1-advanced-first-last-email+additional-646w.png

The Conditions tab provides rules one or more of the records in a duplicate group will need to meet.

  • Value Required in All Records - Each record must contain a value in this field to be considered a duplicate.
  • Empty Allowed in Any Record - A record can still be considered a duplicate if this field is blank. Allowing empty values requires using two or more fields to identify duplicates.
  • At Least One Record With Non-Empty - At least one record in the duplicate group must contain a value.
  • At Least One Record Match - At least one record in the duplicate group must match the specified value. If none of the records have the specified value, the duplicate group will not be merged.
  • Only One Record Match - If more than one record in a duplicate group contains the specified field value, the duplicate group is skipped (not merged).
  • Within Timeframe - Set a time parameter that can find duplicates created or modified within a specific timeframe, such as the last 30 minutes.
  • Values Don't Match - All records in the duplicate group must have a value, and it doesn't match the other records within the group.

merge-duplicates-salesforce-contacts-step-1-conditions-all-5.png

Analyze the Identified Duplicates

Records that have the same values in the fields specified in Step 1 are considered matches. When two or more records represent the same entity (person, company, or other object type), they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates. For example, if you had four records for the same person, it would count as one duplicate group with four records.

Click the chevron   at the right end of a duplicate group row to expand it and examine the individual records within that group. 

To exclude specific records from the merge in Manual mode, uncheck the checkbox next to any record you want to keep separate. Only the checked records will be merged — unchecked records will remain untouched in your CRM. For example, if a group contains four duplicate contacts and you only want to merge three of them, uncheck the one you want to keep separate before proceeding to Step 3.

If you are using Bulk mode, individual record deselection is not available. Exclusions are managed through your Step 1 matching criteria and filters instead. See Can I exclude or keep separate specific records within a duplicate group? for details.

merge-duplicates-contacts-manual-mode-step-2-tasneem-selected.png

Choose Manual Mode and Select the Records to Merge

Under Step 3. Merge Logic, select Manual mode to have complete control over which records are merged. You'll work with the single duplicate group you've chosen throughout the entire merge process.

merge-duplicates-step-3-manual.png

Select the Master Record

The master is the record that will remain after the duplicates are merged. If you select three records and merge them, the other two will no longer exist. By default, data from your chosen master will be retained, and any blank values will be automatically filled from the other records. If you'd like more control over the data saved in the master record, you can do so later, in Step 4.

Under Step 3, select the master record into which the other records will be merged. It is usually best to select the record with the email address you want to keep.

merge-duplicates-hubspot-contacts-step-3-manual-master-selected.png

Merge API and Blank Fields (HubSpot and Salesforce Only)

When working with HubSpot or Salesforce, two additional settings appear at the bottom of Step 3: Merge API and Blank Fields. These control how Insycle executes the merge and whether empty master fields are automatically filled.

Merge API determines which merge logic Insycle uses to combine records.

  • Native uses your CRM's built-in merge logic. In HubSpot, this uses HubSpot's native merge behavior. In Salesforce, this uses Salesforce's Apex API merge logic.
  • Synthetic uses Insycle's custom merge logic, which gives you more control over field retention and master selection than your CRM's native merge provides. Synthetic is the only option available for object types that don't have native merge support in your CRM.

The default Merge API selection depends on your platform and object type. See the Merge Duplicates Module Overview for the full defaults table and details on Native vs. Synthetic merge behavior.

Blank Fields controls whether Insycle automatically fills empty fields in the master record with values from duplicates.

  • Enabled (toggled yellow) — Insycle automatically fills empty master record fields with values from the most recently updated duplicate where that data exists. This is the default setting and is recommended for most use cases.
  • Disabled (toggled grey) — Empty fields in the master record remain empty after the merge. No automatic filling occurs.

Choose or Edit Values to Merge into the Master, then Merge

If you want to control which values are kept, you can choose or edit specific fields under Step 4. This is an optional step—if you don’t pick specific values, the default merging logic will be followed. For HubSpot and Salesforce users, the Merge API setting (Native or Synthetic) and the Blank Fields control operate independently from field-level value selection. Regardless of which Merge API is selected, you can still manually select or edit individual field values here in Step 4. Learn more about what happens to field data that isn't in the master record in the FAQs below.

Under Step 4, only five fields are initially displayed. To see all the fields, change the number of rows shown per page.

merge-duplicates-step-4-manual-merge-rows-per-page.png

There are three tabs showing different field sets:

  • The Conflicts tab shows only the fields that have differences in the values, making it easy to focus only on fields that need attention
  • The Read Only tab lists only the non-writable fields from your database
  • The Full tab shows all of the record fields, even those without data

Use the search to find specific fields.

On a field-by-field basis, select which values to keep and merge into the master. For example, you could choose the Industry from one and City from another. 

merge-duplicates-contacts-manual-mode-step-4-tasneem-choose-values.png

To edit the individual field values one by one, hover over a value you want to update and click the pencil icon.

merge-duplicates-contacts-manual-mode-step-4-tasneem-edit-pencil.png

In the Edit popup, update the value and click Save

merge-duplicates-contacts-manual-mode-step-4-tasneem-edit-save.png

Apply the Merge to Your CRM

When you have all critical fields chosen, click the Merge button at the bottom to merge your selected duplicates. 

⚠️ Note that there is no preview; once you confirm, this change is applied directly to your CRM.
merge-duplicates-contacts-manual-mode-step-4-tasneem-merge.png

The Merge Result popup provides a summary of the merge operation results.

merge-duplicates-contacts-manual-mode-step-4-tasneem-result.png

Emailed CSV Report, Audit Trail, and History

Once you've run the merge operation, Insycle will generate a CSV report and send it to your email. You can use this to review the changes made through Insycle.

merge-duplicates-contacts-manual-mode-tasneem-csv.png

With the Activity Tracker, you can download a CSV report at any time that shows all the changes made in a given operation run.

Navigate to Operations > Activity Tracker, search by module, app, or template name. The operations run in Manual Mode are indicated by the Manual checkbox. Click the Run ID for the operation to download the CSV report.

activity-tracker-hubspot-merge-duplicate-contacts-run-id+manual.png

When To Use Manual vs. Bulk Merging

Manual merging is the right choice when you have a small number of duplicates to address, need granular control over each merge, or want to review records individually before making changes. It's also useful when you're exploring your duplicate data for the first time to understand what you have and how best to approach it.

Bulk merging is recommended for most deduplication work, especially with larger datasets. Bulk mode automatically merges all duplicate groups according to the master record selection rules you define, and is the only mode that supports templates, Recipes, and automation.

For situations where you have a large number of records to merge but no common rules you can apply for identifying duplicates, you can create a CSV and use the Magical Import module to customize your bulk merging, designate master records, and exclude records from deduplication. Learn how to customize merging duplicates in bulk using a CSV.

For a step-by-step walkthrough of bulk deduplication, see Bulk Merge Duplicate People and Companies.

Explore other advanced approaches to merging duplicates in our Deduplication Scenarios article.

Troubleshooting

Insycle isn't finding any duplicates

Most of the time, when Insycle can't find duplicates, it is due to your matching rules in Step 1. It is important to analyze the underlying data to better understand how to set up your rules. A helpful exercise for contacts is to set up your matching filters to find exact matches on only First Name and Last Name. For companies, you can use Company Name.

step-1-fname-Lname-only.png

When you click Find, these rules can show you a broad overview of what duplicates are potentially in your database and what fields might be useful to include in your matching fields. These settings are just for discovery and should not be used for a final merge operation; many people can have the same first and last names and are not duplicates. 

To get further context, on Step 2, click the layout gear icon-gear-18x18.png button on the right side of the title bar. Here, you can add any field from your database as a column in the duplicate group review to better understand the data in these records. These will also appear in your CSV report. 

merge-duplicates-step-2-gear-arrow-646w.png

It's taking a long time for Insycle to find duplicates

It can take Insycle a while to find and match duplicates when the fields used to identify them have very long values. The longer the values, the longer it takes Insycle to process the data and generate the results. This might come up when looking for matches based on long ID numbers, LinkedIn bio links, or other URLs with long strings attached (ex, https://www.linkedin.com/in/svadin%C3%ADr-n%C4%9Bmec-1234b31a3/).

You can speed this up by limiting how much of the value Insycle looks at.

If the beginning or ending portion of the values are all unique, you can limit the comparison to the first or last several characters using the Match Parts parameter under Step 1

merge-duplicates-linkedin-URL-step-1-match-parts-last-9-chars-646w.png

merge-duplicates-linkedin-bio-step-2-last-9-chars.png

Or use the Ignore Text (Substrings) parameter, then click the Terms button.

merge-duplicates-linkedin-URL-step-1-ignored-text-terms-button-646w.png

On the Ignored Text tab of the popup, add the common portion of the URL or text string.

merge-duplicates-linkedin-URL-step-1-ignored-text-popup-646w.png

For a complete guide to troubleshooting issues with Insycle, please refer to our article on Troubleshooting Issues.

Frequently Asked Questions

What happens to field data that isn't in the master record? Will I lose data?

When manually merging duplicates, Insycle will, by default, keep the master record values. If a master record has empty fields, Insycle copies values from the most recently updated record in the duplicate group where that data exists. Any data that is not in the master or not copied to the master is removed.

merge-duplicates-most-recent-value-ani-644x374.gif

For example, if the master record's Industry field is empty but another record in the duplicate group has an Industry value, that value will automatically be copied to the master record.

Additionally, you can manage which values are saved in the master record after the merge, regardless of the default merge behavior. 

Select each field value you wish to retain under Step 4 and click the Full tab. The Full tab shows all of the record fields, even those without data. On a field-by-field basis, select which values to keep and merge into the master. 

For example, you could choose the Primary Associated Company ID from one and Company Name from another. 

merge-duplicates-hubspot-contacts-manual-step-4-full-tab-choose-values-646px.png

Can I exclude or keep separate specific records within a duplicate group?

Yes, you can exclude or keep specific records separate within a duplicate group, though the approach depends on whether you are working in Manual mode or Bulk mode.

Manual mode:

Under Step 2, expand the duplicate group by clicking the chevron at the right end of the group row. Then uncheck the checkbox next to any record you want to keep separate — only the checked records will be merged, and unchecked records will remain untouched in your CRM.

For example, if Insycle identifies a group of four duplicate contacts and you only want to merge three of them, uncheck the one you want to keep separate before proceeding.

merge-duplicates-contacts-manual-mode-step-2-tasneem-selected.png

Bulk mode: 

In Bulk mode (Step 3 set to Bulk), you can't deselect individual records. Instead, exclusions are handled through your Step 1 matching criteria and filters, which determine which records are considered duplicates in the first place. Options include:

  • Filter by a unique field value in Step 1 to exclude it — Click the Filter button in Step 1 and select the field, such as Record ID, in the popup. Set the condition to "is not" [that record's ID] to prevent that specific record from appearing in the duplicate group. See Bulk Merge Duplicate People, Companies for details.

    merge-duplicates-step-1-filter-popup-record-ID-is-not-646w.png
  • Add a field to your matching criteria in Step 1 — If a field value distinguishes the record you want to exclude (such as object type, owner, or status), click the +Field button to add it as an additional matching criterion so that the record no longer qualifies as part of the duplicate group.

    merge-duplicates-contacts-step-1-email-status-exact-match-646w.png
  • Use a dedicated exclusion field — For ongoing exclusions across multiple templates, consider creating a custom field in your CRM (e.g., a checkbox labeled "Deduplication Exclude"). Flag records that should never be merged, then add a Step 1 filter for that field "is not true." Learn more in Customize How Duplicates Are Merged Using a CSV.

    merge-duplicates-step-1-filter-deduplication-exclude-not-yes-646w.png

What is the Merge API in Step 3?

The Merge API options are unique features for HubSpot and Salesforce accounts. The Native merge uses your CRM's built-in merge functionality, while the Synthetic merge provides Insycle's unique merging capabilities. Learn more about the Merge API for HubSpot or the Merge API for Salesforce.

merge-duplicates-hubspot-leads-manual-step-3.png

Can I view more fields to help me analyze my duplicates?

To help you analyze and determine which records to merge, you can change the fields that appear in the preview under Step 2 by clicking the icon-gear-18x18.png gear button to adjust the layout.

merge-duplicates-step-2-gear-arrow.png

Additional Resources

Related Help Articles

Related Blog Posts