Deduplicate Across Salesforce Leads and Contacts

Navigate to Data Management > Merge Duplicates, and select the Salesforce database and Contacts record type from the top menu. Then explore the templates for an existing solution that may be close to what you need.

In Step 1, you define what criteria to use to identify records as duplicates. 

1. Click the + Field button to add each field you want to compare to find a matching value. Choose fields that, in combination, give a high degree of certainty that the matched records are duplicate records.

2. Check the Include Leads checkbox. With the box checked, Insycle will identify and merge duplicate leads and contacts into a single record.

3. Click the Find button, and Insycle will generate a list of duplicates for you to review.

For complete details on configuring duplicate-matching rules, see the Setting the Criteria for Finding Duplicate Matches article.

Expand Criteria for Matching Duplicates

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab. For example, look at both the Mobile Phone and Business Phone fields for duplicate values.

The Conditions tab provides rules that one or more of the records in a duplicate group must meet. These options let you choose required fields, fields that can be empty, or specify values that must be included.

Records that have the same values in the fields from Step 1 are considered matches, whether they are all leads, all contacts, or a mix. When two or more records represent the same person based on your matching rules, they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates based on your settings. For example, if you had four records for the same person, it would count as one duplicate group with four duplicate records.

Under Step 2, each row represents a duplicate group, with the number indicating the number of duplicate records found. The Type column indicates whether the duplicate record is a contact or a lead. When you click the chevron   at the right end of a duplicate group row to expand it and examine the individual records within that group. 

To exclude a duplicate group from all future deduplication analysis, click the X on the group row. Learn more about adding and removing duplicate groups from the Exclusions list in the Advanced How-Tos below.

Explore the record data in the duplicate groups. Double-check to ensure that the fields you set up in Step 1 show what you expected.

Looking at the record details in several duplicate groups, decide the best way to determine which is the master record that the other records will be merged into. 

To get more context, add more columns to the view using the gear button at the right end of the header.

In Step 3, you define how your duplicates are merged. You'll choose your operation mode, set rules for which record becomes the master, and optionally control which field values are retained after the merge. 

Choose Bulk Mode

The most efficient and sustainable way to merge duplicates is in Bulk mode. This allows you to set rules to automatically determine the master record for all lead and contact records in your database. You can then use saved templates and recipes to repeat the process.

At the top of Step 3, the Bulk option is automatically selected. Leave this as is.

Configure Rules to Automatically Select the Master Record (Master Tab)

On the Master tab of Step 3, define how all of the duplicate groups should be merged at scale. To do this, you need to create a series of rules that specify which record from each group should become the master. The master is the record that will remain after the merge.

For example, if you had four records representing the same person in one duplicate group, all four would be merged into one master record, and the other three records would no longer exist.

For each duplicate group, Insycle will look at each Master rule to see which records in the group meet the criteria. Rules are read in order, from top to bottom, and as soon as a record is the only one to meet a rule, it is selected as the master record. The subsequent rules are ignored. 

Select the processing method—By Priority or Absolute. Most de-duplication operations should use By Priority. When a record meets one of the criteria, Insycle makes the master selection and skips the rest of the rules on the list. 

Learn more about these options in the Choosing Which Records to Merge Using a CSV article.

Configure Rules That Determine Values to Keep (Fields Tab)

On the Fields tab of Step 3, you can create custom data retention rules to pick which values from a duplicate group will be used in the master record. By default, Insycle will keep the most recent values. You can override this field-by-field for any fields that need special handling.

For example, you may want to keep the job title that appears most frequently, or the country from the record that also has city and state data.

For each field you want to control data retention for, select the field and specify where the data should be taken from. This information is merged into the master. Any other data not in the master or copied to the master is deleted.

The Criteria dropdown gives you various options for choosing the data to keep, and the Group Fields let you keep values of multiple fields from the same record.

Learn more about these options in the Keeping the Right Data When Records Merge article.

Customize the Method Settings (Method Tab)

When working with Salesforce, the Method tab provides a couple of additional options.

Merge API: This is the type of merge logic Insycle will use.

• The Native method is the default for Salesforce contacts, leads, and accounts. It uses Salesforce’s APEX API merge logic.
• The Synthetic method is the default for all other object types, including opportunities and custom objects.

Blank Fields: Here, you can disable the automatic fill-in-the-blank feature when there is an empty field in the master record, and prevent values from being copied from other duplicates to the master. This fill-in-the-blank feature is enabled by default (yellow). Use the toggle to disable it (grey).

See Native vs. Custom Merge: Two Ways to Combine Records in HubSpot and Salesforce for details.

Step 4 is where you preview and apply your merge changes.

Preview Merge Changes in CSV Report

Now that the filters and master record are set up, you can preview how the merge would be applied. That way, you can check to ensure your configuration works as expected before the changes are pushed to your live Salesforce database. 

Under Step 4, click the Review button and select Preview mode.

Click the Next button to go to the Notify screen and enter recipients for the email report. You can also add additional context to the email message.

On the When tab, click the Run Now tab, select which records to apply the change to (if you have many records, you might want to test only a few), then click the Run Now button.

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application and review the data.

There's a row for each record, which includes:

• The Result of the action
• A Message with details
• The Duplicate Group ID indicates which records will be merged together
• A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
• All fields used to identify the duplicates in Step 1
• The record ID, record name, or email, and Deeplink to the CRM record
• The Type, indicating whether the record is a Contact or Lead
• All fields selected for the Master and Fields rules in Step 3. Note that if a field is used on both tabs, it will only appear once in the CSV. 
• Any additional fields added to the Layout tab in Step 2

The Status column indicates:

• Duplicate – The record is part of a duplicate group.
• Master – The master record chosen for the duplicate group based on default behavior and your Master rules. Review the selections in this row to determine whether the appropriate records are being chosen.
• Master (After) – For each duplicate group, the Master (After) row shows the values the final record will contain based on your Field rules and the default behavior. 
• Error – If Insycle cannot determine which record to select as master, an error message will appear here. See the Merge Duplicates Troubleshooting article for next steps.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your CRM Records

If everything in your CSV preview looks good, return to Insycle and move forward with applying the changes to your live Salesforce data.

Under Step 4, click the Review button, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to Salesforce.

After you've seen the results in Salesforce and are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a recipe.

By automating with a template, you'll save time and ensure that your leads and contacts are merging consistently on an ongoing basis.

Save Your Configuration as a Template

Return to the Template menu at the top of the page and click Copy to save your configurations as a new version of the template you started with. Then click the pencil to edit your new template name.

Set Up Automation

Under Step 4, click the Review button and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can also provide additional context in the message subject or body.

On the When tab, select Automate and configure the frequency you'd like the template to run. When finished, click Schedule.

You can view all your scheduled automations at any time on the Operations > Automations page.

When you have a solid set of templates that reliably merge your records, you can put them together into a longer, ordered sequence as a Recipe. Then, you can schedule that Recipe to run on a consistent, set schedule. Your templates will run one after another in the order that you set.

To add your Recipe to a Salesforce Flow, the Automate setting must be Salesforce Flow Integration.

Learn more about integrating Insycle Recipes with Salesforce flows.

If your database is large or you're getting an overwhelming number of duplicate groups, use the Filter button in Step 1 to narrow down the records Insycle analyzes. Filtering to a subset — for example, contacts created in the last 30 days, or companies in a specific region — makes it easier to validate your configuration before running it across your full database. A filtered dataset also processes faster.

To add a filter, click the Filter button in Step 1, choose a field, select a condition, and set a value. The filter is applied before the matching step runs.

Most of the options in the Field dropdown match the fields that are found in your HubSpot records, and for contact records, there are three additional options related to the Email value: 

• Email Username: The portion of the email address before the “@.” For example, if the email address were “maria@acmewidgets.com,” the username value would be “maria.” 
• Free Email Provider Domain: Choose True to filter out records where the email domain is Gmail, Hotmail, Yahoo, or any of about 10,000 other free email providers. This filter helps ensure these are real clients, or can determine which record is the legitimate one, because it’s most likely that customer companies aren't using free Gmail accounts (though a contact may have accidentally emailed us from one at some point). 
• Email Top-Level Domain: The top-level domain (TLD) is everything that follows the final dot of a domain name. For example, in the domain name acmewidgets.com, '.com' is the TLD. Some other popular TLDs include '.org', '.uk', and '.edu'. 

When setting up your matching fields for the first time, start with your highest-confidence criteria — fields that, together, are very unlikely to match unless the records are true duplicates. Exact Match on email address plus first and last name is a reliable starting point for contacts. Domain Name is a strong starting point for companies.

Once you're confident your configuration is catching true duplicates without pulling in false positives, you can broaden your criteria — for example, by adding Similar Match to catch typos and slight variations: 

Or, by using Related Fields to compare values across two fields that contain similar data (such as Email and Additional Email).

In HubSpot, the master record always determines which primary email is kept after a merge. If you want to control which primary email address is kept, set up a rule to select the appropriate master record.

Review the email values in Step 2 to identify the criteria, then set up a Master rule in Step 3 to choose the record for each duplicate group based on that criteria. 

For example, to select the master based on corporate email addresses over personal ones (like choose the record with john.smith@acmecorp.com over john.smith@gmail.com), configure a master rule:

• Field Name: Email
• Condition: Work domain (non-Gmail, etc.)

Other Example Master Rule Configurations:

• Field Name: Email
• Condition: Select from the following options based on your business rules:
  • Personal domain (Gmail, etc.) — Prioritize personal email addresses; useful when merging consumer contacts or individual leads.
  • Not role-based (info@, etc.) — Exclude generic email addresses like info@company.com or sales@company.com to keep only personal contact emails.
  • Is / Is not — Match or exclude an exact value. For example, use "Is not" with "noreply@company.com" to avoid selecting automated email addresses as master records.
  • Contains / Does not contain — Match or exclude based on partial text. For example, use "Contains" with "@acmecorp.com" to prioritize all company domain emails, or "Does not contain" with "+test" to exclude test email addresses.
  • Regex — Use a regular expression pattern for complex matching. For example, use a regular expression (regex) pattern to match emails from multiple specific domains or to identify specific naming patterns.
  • Exists / Doesn't exist — Filter based on whether the field has any value. For example, use "Exists" to prioritize records with an email address over those without one.

Retaining Other Email Data

Though you cannot use the Fields rules to control what HubSpot primary email field data is retained, you can set retention rules to copy non-master email values to a different field, such as Additional Email or a custom field.

For example, to preserve non-master email values as backup contact information, configure a field rule:

• Field Name: Additional email addresses
• Criteria: Collect non-master values from
• Other Field: Email

By default, all duplicate groups identified in Step 1 are candidates for merging. If there are specific groups you want to permanently prevent from being merged — regardless of which matching rules or templates you use — you can add them to the Duplicate Groups Exclusion List directly in Step 2.

 Note that this feature applies only to Bulk mode. If you are using Manual mode in Step 3, use the record checkboxes to deselect individual records instead.

To exclude a duplicate group:

In Step 2, click the X on a duplicate group row. The group will be added to the Exclusion List and will no longer appear in duplicate analysis or be included in merges, even when using different duplicate detection templates.

To review and manage excluded groups:

Click the Exclusions button in the Step 2 header. This opens a dialog showing all excluded groups. 

You can expand each group to view the individual records and their values, and use the Layout tab to configure which fields are displayed. To remove a group from the Exclusion List, click the X next to it — those records will once again be considered during duplicate analysis.

Important considerations:

• Exclusions apply to Bulk mode only and to the entire duplicate group — they cannot be applied to individual records within a group. If you need to exclude a single record rather than the entire group, switch to Manual mode in Step 3 and uncheck the records that should not be merged.
• Exclusions are based on a specific set of record IDs. If a new duplicate record is later added to your CRM, the system may detect a new group containing that record alongside previously excluded records. Because the group now includes a different set of record IDs, it will appear again during duplicate analysis and can be evaluated or excluded as needed.

Insycle evaluates your Master tab rules from Step 3 in order, eliminating records that don't match each rule until only one remains. If multiple records still match after all rules are evaluated, no master can be determined, and the group will show an error in the CSV.

For example, imagine having four duplicate records for the same contact. (In this image, we are examining the records in the Grid Edit module.)

In the Merge Duplicates module, you have configured the first three Master rules based on email engagement metrics, but all four records have identical values of zero, so no records are eliminated. Your fourth rule checks for an active Contact Owner — three records have no owner, so they're eliminated. The one remaining record becomes the master.

This is why rule order matters. Place your most reliably differentiating rules — like record owner, lifecycle stage, or engagement activity — where they're most likely to yield a clear winner. If you're frequently seeing errors in your CSV, revisit your rule order and consider adding a tiebreaker rule, such as earliest Create Date or latest Last Modified Date, as a final fallback.

In this CSV report example, you can see that the one record with an active owner was chosen as the master.

Even though this was the fifth rule, it is the first one that matched only one record:

• When Insycle processed the first three rules—Marketing emails clicked, emails bounced, and emails opened—Insycle couldn't eliminate any records because they all had a value of 0.

• None of the record email addresses are "role-based," so there isn't a single choice for the master using the Email rule either.

• In the next rule about contact owner, there is one record with an active contact owner; therefore, that is the master record.

For situations where there are no common rules you can apply for identifying duplicates for all or some of your records, you may need more granular control over which records are included or excluded from the process.

Bulk Solutions

There are two options for doing this in bulk. You can upload a CSV file of known duplicate record ID pairs directly into the Merge Duplicates module using the CSV tab in Step 1. This lets you bypass field-based duplicate detection entirely and work from a list of specific record pairs you've already identified. From there, you continue through Steps 2–4 to configure master selection rules and field retention settings, and then run the merge. 

For more complex scenarios — such as designating master records or excluding specific records from deduplication using custom attributes — you can also use the Magical Import module in combination with Merge Duplicates for complete control over the process. 

Learn how to customize merging duplicates in bulk using a CSV.

Single Record Solution

To do this one record at a time, you can use Manual mode of the Merge Duplicates module.

In Manual mode, you have complete control over which records are merged together by selecting them from the Record Viewer. Manual mode should be reserved for cases that require a careful, controlled process. Learn more about merging duplicates in Manual mode.

When setting up your merge operation in the Merge Duplicates module, add additional fields to the CSV report so you have the data later for undoing changes and/or general review.

Under Step 2, click the gear icon in the header.

On the Layout tab, add any extra fields to the Visible Fields list.

The fields will be included in the CSV report.

Before running a large merge operation, you can export all records directly from Salesforce to a CSV file. This can preserve all record details, which can help later for undo and/or general review.

Create a Targeted Salesforce Report

To select only the necessary fields and filter out inapplicable records, you can create a Salesforce Report. This gives you a targeted export that omits unnecessary data. Once configured and saved, you can reuse this report before each bulk merge operation.

In Salesforce, if the Reports tab doesn't appear at the top, click the App Launcher icon (nine squares) in the top left corner and search for "Reports."

Create a new report based on the specified data type and include all desired fields. Add filters to narrow the results. Save and Run the report.

On the results page, click the arrow next to the Edit button and select Export. In the Export popup, select Details and CSV as the format.

Export All Data for a Record Type

To get all field data for all records of the same type, you can use Salesforce's Data Export Service. There are no filtering/segmenting options, so if you have a large dataset, this can take a while to run.

In Salesforce, navigate to Setup > Data > Data Export.

Click Export Now, select the record type, and then click Start Export. You'll receive an email when the file is ready for download.