Deduplicate Across Salesforce Leads and Contacts

Deduplicate leads and contacts

How to Merge Duplicate Individuals Looking at Both Leads and Contacts

Your company is experiencing issues because the same person appears in Salesforce as both a lead and a contact. However, looking for duplicates in the combined pool of leads and contacts can't be done using native Salesforce features.

Insycle's Merge Duplicates module can identify duplicate records and merge them flexibly, in bulk, using advanced rules and master record selection logic. You can deduplicate across Salesforce leads and contacts simply by clicking a checkbox.

Process Summary

  1. Set rules to identify duplicates.
  2. Check the Include Leads checkbox.
  3. Review and analyze the identified duplicates.
  4. Choose Bulk Mode.
  5. Set master record selection and rules determining what field data is retained.
  6. Preview the changes and deduplicate your records in Salesforce.

 

Step-by-Step Instructions

Step 1: Set Rules to Identify Duplicates across Contacts and Leads

Navigate to Data Management > Merge Duplicates, and select the Salesforce database and Contacts record type from the top menu. Then explore the templates for an existing solution that may be close to what you need.

In Step 1, you define what criteria to use to identify records as duplicates. 

  1. Click the + Field button to add each field you want to compare to find a matching value. Choose fields that, in combination, give a high degree of certainty that the matched records are duplicate records.

  2. Check the Include Leads checkbox. With the box checked, Insycle will identify and merge duplicate leads and contacts into a single record.

  3. Click the Find button, and Insycle will generate a list of duplicates for you to review.

For complete details on configuring duplicate-matching rules, see the Duplicate Identification Rules reference in the Module Overview.

merge-duplicates-salesforce-contacts-step-1-simple-tab-first-last-domain-include-leads-646w.png

Expand Criteria for Matching Duplicates

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab. For example, look at both the Mobile Phone and Business Phone fields for duplicate values.

merge-duplicates-salesforce-contacts-step-1-advanced-tab-related-field-phone-w-arrow-646w.png

The Conditions tab provides rules that one or more of the records in a duplicate group must meet. These options let you choose required fields, fields that can be empty, or specify values that must be included.

For complete details, see the Merge Duplicates Module Overview.

merge-duplicates-salesforce-contacts+leads-step-1-conditions-all-6.png

Step 2: Analyze the Identified Duplicates

Records that have the same values in the fields from Step 1 are considered matches, whether they are all leads, all contacts, or a mix. When two or more records represent the same person based on your matching rules, they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates based on your settings. For example, if you had four records for the same person, it would count as one duplicate group with four duplicate records.

Under Step 2, each row represents a duplicate group, with the number indicating the number of duplicate records found. The Type column indicates whether the duplicate record is a contact or a lead. When you click the chevron   at the right end of a duplicate group row to expand it and examine the individual records within that group. 

To exclude a duplicate group from all future deduplication analysis, click the X on the group row. Learn more about adding and removing duplicate groups from the Exclusions list in the Advanced How-Tos below.

step-2-salesforce-leads&contacts.png

Explore the record data in the duplicate groups. Double-check to ensure that the fields you set up in Step 1 show what you expected.

Looking at the record details in several duplicate groups, decide the best way to determine which is the master record that the other records will be merged into. Learn common practices for picking a master record.

To get more context, add more columns to the view using the icon-gear-18x18.png gear button at the right end of the header.

merge-duplicates-step-2-gear-arrow.png

Step 3: Configure Merge Logic

In Step 3, you define how your duplicates are merged. You'll choose your operation mode, set rules for which record becomes the master, and optionally control which field values are retained after the merge. 

Choose Bulk Mode

The most efficient and sustainable way to merge duplicates is in Bulk mode. This allows you to set rules to automatically determine the master record for all lead and contact records in your database. You can then use saved templates and recipes to repeat the process.

At the top of Step 3, the Bulk option is automatically selected. Leave this as is.

merge-duplicates-hubspot-contacts-step-3-bulk-manual-mode-highlight-646w.png

Configure Rules to Automatically Select the Master Record (Master Tab)

On the Master tab of Step 3, define how all of the duplicate groups should be merged at scale. To do this, you need to create a series of rules that specify which record from each group should become the master. The master is the record that will remain after the merge.

For example, if you had four records representing the same person in one duplicate group, all four would be merged into one master record, and the other three records would no longer exist.

For each duplicate group, Insycle will look at each Master rule to see which records in the group meet the criteria. Rules are read in order, from top to bottom, and as soon as a record is the only one to meet a rule, it is selected as the master record. The subsequent rules are ignored. 

Select the processing method—By Priority or Absolute. Most de-duplication operations should use By Priority. When a record meets one of the criteria, Insycle makes the master selection and skips the rest of the rules on the list. 

Learn more about these options in the Master Selection Rules reference in the Module Overview article.

merge-duplicates-salesforce-contacts+leads-step-3-master-tab-5-rules-646w.png

Configure Rules That Determine Values to Keep (Fields Tab)

On the Fields tab of Step 3, you can create custom data retention rules to pick which values from a duplicate group will be used in the master record. By default, Insycle will keep the most recent values. You can override this field-by-field for any fields that need special handling.

For example, you may want to keep the job title that appears most frequently, or the country from the record that also has city and state data.

For each field you want to control data retention for, select the field and specify where the data should be taken from. This information is merged into the master. Any other data not in the master or copied to the master is deleted.

The Criteria dropdown gives you various options for choosing the data to keep, and the Group Fields let you keep values of multiple fields from the same record.

Learn more about these options in the Field Data Retention Rules reference in the Module Overview article.

merge-duplicates-salesforce-contacts-step-4-fields-tab-10-rules-cropped.png

What happens to field data if I don’t create custom rules?

There is no need to create rules for every field in your CRM - Insycle automatically handles fields without specific rules using a "fill in the blanks" approach. When the master record has empty fields, Insycle copies values from the most recently updated record in the duplicate group where that data exists. For example, if the master record's Industry field is empty, but another record in the duplicate group has an Industry value, that value will automatically be copied to the master record. This means you only need to create custom retention rules for the handful of fields that require special handling rather than setting up rules for all your fields. To override this, disable the Blank Fields autofill setting on the Method tab.

Customize the Method Settings (Method Tab)

When working with Salesforce, the Method tab provides a couple of additional options.

Merge APIThis is the type of merge logic Insycle will use.

  • The Native method is the default for Salesforce contacts, leads, and accounts. It uses Salesforce’s APEX API merge logic.
  • The Synthetic method is the default for all other object types, including opportunities and custom objects.

Blank Fields: Here, you can disable the automatic fill-in-the-blank feature when there is an empty field in the master record, and prevent values from being copied from other duplicates to the master. This fill-in-the-blank feature is enabled by default (yellow). Use the toggle to disable it (grey).

merge-duplicates-salesforce-contacts-step-4-method-tab-native-fill-on-646w.png

Step 4: Review CSV Report and Merge in Salesforce

Step 4 is where you preview and apply your merge changes.

Preview Merge Changes in CSV Report

Now that the filters and master record are set up, you can preview how the merge would be applied. That way, you can check to ensure your configuration works as expected before the changes are pushed to your live Salesforce database. 

Under Step 4, click the Review button and select Preview mode.

merge-duplicates-step-4-review-preview-mode-tab.png

Click the Next button to go to the Notify screen and enter recipients for the email report. You can also add additional context to the email message.

On the When tab, click the Run Now tab, select which records to apply the change to (if you have many records, you might want to test only a few), then click the Run Now button.

merge-duplicates-step-4-review-preview-run-now-custom-646w.png

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application and review the data.

There's a row for each record, which includes:

  • The Result of the action
  • A Message with details
  • The Duplicate Group ID indicates which records will be merged together
  • A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
  • All fields used to identify the duplicates in Step 1
  • The record ID, record name, or email, and Deeplink to the CRM record
  • The Type, indicating whether the record is a Contact or Lead
  • All fields selected for the Master and Fields rules in Step 3. Note that if a field is used on both tabs, it will only appear once in the CSV. 
  • Any additional fields added to the Layout tab in Step 2

Preview merge results in CSV

The Status column indicates:

  • Duplicate – The record is part of a duplicate group.
  • Master – The master record chosen for the duplicate group based on default behavior and your Master rules. Review the selections in this row to determine whether the appropriate records are being chosen.
  • Master (After) – For each duplicate group, the Master (After) row shows the values the final record will contain based on your Field rules and the default behavior. 
  • Error – If Insycle cannot determine which record to select as master, an error message will appear here. See the Troubleshooting section below for more details.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your CRM Records

If everything in your CSV preview looks good, return to Insycle and move forward with applying the changes to your live Salesforce data.

Under Step 4, click the Review button, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to Salesforce.

merge-duplicates-step-4-review-update-run-now-chunk-646w.png

Save Templates and Setup Automation

After you've seen the results in Salesforce and are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a recipe.

By automating with a template, you'll save time and ensure that your leads and contacts are merging consistently on an ongoing basis.

Save Your Configuration as a Template

Return to the Template menu at the top of the page and click Copy to save your configurations as a new version of the template you started with. Then click the pencil to edit your new template name.

save-template-copy-and-rename.png

Set Up Automation

Under Step 4, click the Review button and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can also provide additional context in the message subject or body.

merge-duplicates-step-5-update-notify-tab-always-send.png

On the When tab, select Automate and configure the frequency you'd like the template to run. When finished, click Schedule.

merge-duplicates-step-4-review-update-automate-daily.png

You can view all your scheduled automations at any time on the Operations > Automations page.

Create a Recipe and Integrate with Salesforce Flows

When you have a solid set of templates that reliably merge your records, you can put them together into a longer, ordered sequence as a Recipe. Then, you can schedule that Recipe to run on a consistent, set schedule. Your templates will run one after another in the order that you set.

recipe-templates-2-deduplicate-contacts-basic-646w.png

To add your Recipe to a Salesforce Flow, the Automate setting must be Salesforce Flow Integration.

Learn more about integrating Insycle Recipes with Salesforce flows.

recipe-review-update-automate-salesforce-flow-646x454.png

How Merging Works for Leads and Contacts

Insycle uses the Salesforce Apex convert API to merge leads into contacts. This differs from merging leads into leads or contacts into contacts, where Insycle uses the Salesforce Apex merge API.

Advanced How-Tos

Use a filter to work with a subset of your data

If your database is large or you're getting an overwhelming number of duplicate groups, use the Filter button in Step 1 to narrow down the records Insycle analyzes. Filtering to a subset — for example, contacts created in the last 30 days, or companies in a specific region — makes it easier to validate your configuration before running it across your full database. A filtered dataset also processes faster.

To add a filter, click the Filter button in Step 1, choose a field, select a condition, and set a value. The filter is applied before the matching step runs.

dup10.png

Most of the options in the Field dropdown match the fields that are found in your HubSpot records, and for contact records, there are three additional options related to the Email value: 

  • Email Username: The portion of the email address before the “@.” For example, if the email address were “maria@acmewidgets.com,” the username value would be “maria.” 
  • Free Email Provider Domain: Choose True to filter out records where the email domain is Gmail, Hotmail, Yahoo, or any of about 10,000 other free email providers. This filter helps ensure these are real clients, or can determine which record is the legitimate one, because it’s most likely that customer companies aren't using free Gmail accounts (though a contact may have accidentally emailed us from one at some point). 
  • Email Top-Level Domain: The top-level domain (TLD) is everything that follows the final dot of a domain name. For example, in the domain name acmewidgets.com, '.com' is the TLD. Some other popular TLDs include '.org', '.uk', and '.edu'. 

Start narrow, then broaden

When setting up your matching fields for the first time, start with your highest-confidence criteria — fields that, together, are very unlikely to match unless the records are true duplicates. Exact Match on email address plus first and last name is a reliable starting point for contacts. Domain Name is a strong starting point for companies.

merge-duplicates-hubspot-companies-step-1-similar-name-domain-exact-phone.png

Once you're confident your configuration is catching true duplicates without pulling in false positives, you can broaden your criteria — for example, by adding Similar Match to catch typos and slight variations: 

merge-duplicates-hubspot-companies-step-1-comparison-rules-646w.png

Or, by using Related Fields to compare values across two fields that contain similar data (such as Email and Additional Email).

merge-duplicates-hubspot-contacts-step-1-advanced-first-last-email+additional-646w.png

Select the master record based on email

In HubSpot, the master record always determines which primary email is kept after a merge. If you want to control which primary email address is kept, set up a rule to select the appropriate master record.

Review the email values in Step 2 to identify the criteria, then set up a Master rule in Step 3 to choose the record for each duplicate group based on that criteria. 

For example, to select the master based on corporate email addresses over personal ones (like choose the record with john.smith@acmecorp.com over john.smith@gmail.com), configure a master rule:

  • Field Name: Email
  • Condition: Work domain (non-Gmail, etc.)
merge-duplicates-hubspot-contacts-step-3-master-tab-email-646w.png

Other Example Master Rule Configurations:

  • Field Name: Email
  • Condition: Select from the following options based on your business rules:
    • Personal domain (Gmail, etc.) — Prioritize personal email addresses; useful when merging consumer contacts or individual leads.
    • Not role-based (info@, etc.) — Exclude generic email addresses like info@company.com or sales@company.com to keep only personal contact emails.
    • Is / Is not — Match or exclude an exact value. For example, use "Is not" with "noreply@company.com" to avoid selecting automated email addresses as master records.
    • Contains / Does not contain — Match or exclude based on partial text. For example, use "Contains" with "@acmecorp.com" to prioritize all company domain emails, or "Does not contain" with "+test" to exclude test email addresses.
    • Regex — Use a regular expression pattern for complex matching. For example, use a regular expression (regex) pattern to match emails from multiple specific domains or to identify specific naming patterns.
    • Exists / Doesn't exist — Filter based on whether the field has any value. For example, use "Exists" to prioritize records with an email address over those without one.

Retaining Other Email Data

Though you cannot use the Fields rules to control what HubSpot primary email field data is retained, you can set retention rules to copy non-master email values to a different field, such as Additional Email or a custom field.

For example, to preserve non-master email values as backup contact information, configure a field rule:

  • Field Name: Additional email addresses
  • Criteria: Collect non-master values from
  • Other Field: Email
merge-duplicates-hubspot-contacts-step-4-fields-tab-addtl-email-646w.png

Exclude duplicate groups from deduplication 

By default, all duplicate groups identified in Step 1 are candidates for merging. If there are specific groups you want to permanently prevent from being merged — regardless of which matching rules or templates you use — you can add them to the Duplicate Groups Exclusion List directly in Step 2.

To exclude a duplicate group:

In Step 2, click the X on a duplicate group row. The group will be added to the Exclusion List and will no longer appear in duplicate analysis or be included in merges, even when using different duplicate detection templates.

merge-duplicates-hubspot-contacts-step-2-exclude-group-button-646px.png

To review and manage excluded groups:

Click the Exclusions button in the Step 2 header. This opens a dialog showing all excluded groups. 

merge-duplicates-hubspot-contacts-step-2-header-exclusions-button-646px.png

You can expand each group to view the individual records and their values, and use the Layout tab to configure which fields are displayed. To remove a group from the Exclusion List, click the X next to it — those records will once again be considered during duplicate analysis.

merge-duplicates-hubspot-contacts-step-2-manage-exclusions-popup-remove-exclusion-646px.png

Important considerations:

  • Exclusions apply to the entire duplicate group and cannot be applied to individual records within a group. If a group contains some records that are not actually duplicates, use Manual mode in Step 3 and uncheck the records that should not be merged.
  • Exclusions are based on a specific set of record IDs. If a new duplicate record is later added to your CRM, the system may detect a new group containing that record alongside previously excluded records. Because the group now includes a different set of record IDs, it will appear again during duplicate analysis and can be evaluated or excluded as needed.

Understanding how master selection works

Insycle evaluates your Master tab rules from Step 3 in order, eliminating records that don't match each rule until only one remains. If multiple records still match after all rules are evaluated, no master can be determined, and the group will show an error in the CSV.

For example, imagine having four duplicate records for the same contact. (In this image, we are examining the records in the Grid Edit module.)

mceclip11.png

In the Merge Duplicates module, you have configured the first three Master rules based on email engagement metrics, but all four records have identical values of zero, so no records are eliminated. Your fourth rule checks for an active Contact Owner — three records have no owner, so they're eliminated. The one remaining record becomes the master.

This is why rule order matters. Place your most reliably differentiating rules — like record owner, lifecycle stage, or engagement activity — where they're most likely to yield a clear winner. If you're frequently seeing errors in your CSV, revisit your rule order and consider adding a tiebreaker rule, such as earliest Create Date or latest Last Modified Date, as a final fallback.

merge-duplicates-hubspot-contacts-step-3-master-tab-6-rules.png

In this CSV report example, you can see that the one record with an active owner was chosen as the master.

mceclip13.png

Granular control for picking duplicate records

For situations where there are no common rules you can apply for identifying duplicates for all or some of your records, you may need more granular control over which records are included or excluded from the process.

Bulk Solutions

There are two options for doing this in bulk. You can upload a CSV file of known duplicate record ID pairs directly into the Merge Duplicates module using the CSV tab in Step 1. This lets you bypass field-based duplicate detection entirely and work from a list of specific record pairs you've already identified. From there, you continue through Steps 2–4 to configure master selection rules and field retention settings, and then run the merge. 

For more complex scenarios — such as designating master records or excluding specific records from deduplication using custom attributes — you can also use the Magical Import module in combination with Merge Duplicates for complete control over the process. 

Learn how to customize merging duplicates in bulk using a CSV.

Single Record Solution

To do this one record at a time, you can use Manual mode of the Merge Duplicates module.

In Manual mode, you have complete control over which records are merged together by selecting them from the Record Viewer. Manual mode should be reserved for cases that require a careful, controlled process. Learn more about merging duplicates in Manual mode.

Tips for Backing Up Data Before Merging

Backing Up Fields from Insycle

When setting up your merge operation in the Merge Duplicates module, add additional fields to the CSV report so you have the data later for undoing changes and/or general review.

Under Step 2, click the icon-gear-18x18.png gear icon in the header.

merge-duplicates-contacts-step-2-layout-gear.png

On the Layout tab, add any extra fields to the Visible Fields list.

merge-duplicates-hubspot-contacts-job-title-layout-646w.png

The fields will be included in the CSV report.

merge-duplicates-hubspot-contacts-job-title-csv-646w.png

Backing Up Records from Salesforce

Before running a large merge operation, you can export all records directly from Salesforce to a CSV file. This can preserve all record details, which can help later for undo and/or general review.

Create a Targeted Salesforce Report

To select only the necessary fields and filter out inapplicable records, you can create a Salesforce Report. This gives you a targeted export that omits unnecessary data. Once configured and saved, you can reuse this report before each bulk merge operation.

In Salesforce, if the Reports tab doesn't appear at the top, click the App Launcher icon (nine squares) in the top left corner and search for "Reports."

Create a new report based on the specified data type and include all desired fields. Add filters to narrow the results. Save and Run the report.

On the results page, click the arrow next to the Edit button and select Export. In the Export popup, select Details and CSV as the format.

salesforce-reports-backup-contacts.png

Export All Data for a Record Type

To get all field data for all records of the same type, you can use Salesforce's Data Export Service. There are no filtering/segmenting options, so if you have a large dataset, this can take a while to run.

In Salesforce, navigate to Setup > Data > Data Export.

Click Export Now, select the record type, and then click Start Export. You'll receive an email when the file is ready for download. 

salesforce-setup-data-export-contacts.png

Troubleshooting

If you're not seeing the results you expect when merging duplicates, consider these issues:

Not all identified duplicates are merging into the master

You have duplicate records that have been identified by Insycle but not all of them are merging into the master. 

Look at the Exclusions at the bottom of Step 3 to see how many duplicates are in the affected duplicate groups. If you have duplicate groups with more than 5 records, you may want to change the Skip duplicate groups with more than [5] records per group setting to ensure you get all of them.

This 5-record default is intended to prevent accidental merging of non-duplicate records when the filter in Step 1 is too broad.

merge-duplicates-hubspot-contacts-step-3-master-tab-exclusions-w-arrow-646w.png

"Failed" Result in CSV with "Change rules in Step 3 'Master Selection'" Message

If the Result is "Failed" and the Message column of the CSV report displays this text:

Change rules in Step 3 'Master Selection'. Failed to pick master record because multiple records (X) meet the selection criteria. In 'Master Selection', change, add, or reorder the rules such that only one record matches (if cannot determine master based on field values, use 'Record ID is lowest' as the last rule).

merge-duplicates-salesforce-accounts-csv-w-error.png

This means Insycle could not determine which record in the duplicate group should become the master — none of the records meet more of the rules than the others.

There are a few things you can try to resolve this:

  1. Under Step 3, on the Master tab, experiment with reordering or adding rules based on fields that are likely to have unique values across your records. Think about what matters most to your business — the fields that would make one record clearly more valuable or authoritative than another. For example:

    • A field that stores an ID linked to an external system, such as an ERP, data warehouse, or proprietary internal tool — the record with that ID populated is likely the one your other systems depend on
    • The record with the highest number of associated records (contacts, deals, activities) — indicating it has accumulated more history and relationships
    • A field indicating CRM ownership or account status — for example, a record marked as "Customer" vs. "Lead," or one with an active owner assigned
    • A field that tracks revenue, subscription tier, or engagement level — the record with higher values may represent the more complete or current entry
    • A custom field your team uses to flag record quality or data source reliability

    The goal is to surface the fields that reflect record importance for your specific operations, not just default system fields.

  2. Also on the Step 3 Master tab, confirm that By Priority is selected rather than Absolute.

    With By Priority, a record only needs to match one rule to be selected as master. With Absolute, a record must meet all rule criteria, which makes it much harder for any single record to qualify.

    In most cases, By Priority is the right choice. If By Priority is already selected and the error persists, none of the records in the group meet any of the criteria more than the others. Return to the Master tab and continue adding or reordering rules, focusing on the business-critical fields described in #1 above.

    merge-duplicates-step-3-by-priority-match-w-arrow-646w.png

  3. As a last resort, add a rule on the Master tab that uses Record ID is lowest, or Create Date is earliest. These rules will always produce a unique winner, ensuring master selection can complete even when other field-based rules cannot differentiate the records. merge-duplicates-hubspot-contacts-step-3-master-tab-last-resort-rules.png

Insycle isn't finding any duplicates

Most of the time, when Insycle can't find duplicates, it is due to your matching rules in Step 1. It is important to analyze the underlying data to better understand how to set up your rules. A helpful exercise for contacts is to set up your matching filters to find exact matches on only First Name and Last Name. For companies, you can use Company Name.

step-1-fname-Lname-only.png

When you click Find, these rules can show you a broad overview of what duplicates are potentially in your database and what fields might be useful to include in your matching fields. These settings are just for discovery and should not be used for a final merge operation; many people can have the same first and last names and are not duplicates. 

To get further context, on Step 2, click the layout gear icon-gear-18x18.png button on the right side of the title bar. Here, you can add any field from your database as a column in the duplicate group review to better understand the data in these records. These will also appear in your CSV report. 

merge-duplicates-step-2-gear-arrow-646w.png

Non-duplicate records are being merged together

There are a couple of things to look at that may be causing records to be misidentified as duplicates.

First, you may need a better unique identifier. Under Step 1, if you use only fields that can correctly contain the same values across multiple records, they aren't unique identifiers. In this case, you are likely to identify unrelated records as duplicates and may accidentally merge them.

merge-duplicates-intercom-step-1-name-only-646w.png

Unique identifiers are data unlikely to be shared by any other record unless they represent the same underlying entity. Common fields used for deduplication include phone numbers, email addresses, mailing addresses, and ID numbers.

Second, this may indicate the Comparison Rule under Step 1 is too broad. Try using the Exact Match comparison rule instead of the Similar Match rule. Similar Match looks for values that may be close, even with a one-character difference (e.g., a typo), broadening the search. 

Remember to always run your deduplication in Preview Mode to confirm it's working as expected before running it in Update Mode and applying the changes to your CRM records.

Some fields are missing in the Fields tab dropdown in Step 3

Field rules in Step 3 can only update writable fields. Read-only fields are not shown in the Fields tab dropdown because the CRM prevents these properties from being written back. Insycle excludes them from the field selection options to prevent errors.

While you cannot create field rules for read-only properties in the Fields tab, you can use them in the Master tab. Read-only fields, such as "Last Activity Date," can be used to make merge decisions without attempting to modify those properties.

You can also use read-only properties for filtering and matching in Step 1. Read-only fields are also available when reviewing data in Step 2 or in the CSV report.

To identify which fields are read-only, use the Cleanse Data module. Navigate to Data Management > Cleanse Data, then select a database and record type from the top menu. You can search for a specific field by name or click the Writable heading in the table to sort and review all writable or non-writable fields. A checked box in the Writable column indicates the field is writable.

cleanse-data-salesforce-contacts-date-fields-writable-column-arrow-646px.png

It's taking a long time for Insycle to find duplicates

It can take Insycle a while to find and match duplicates when the fields used to identify them have very long values. The longer the values, the longer it takes Insycle to process the data and generate the results. This might come up when looking for matches based on long ID numbers, LinkedIn bio links, or other URLs with long strings attached (ex, https://www.linkedin.com/in/svadin%C3%ADr-n%C4%9Bmec-1234b31a3/).

You can speed this up by limiting how much of the value Insycle looks at.

If the beginning or ending portion of the values are all unique, you can limit the comparison to the first or last several characters using the Match Parts parameter under Step 1

merge-duplicates-linkedin-URL-step-1-match-parts-last-9-chars-646w.png

merge-duplicates-linkedin-bio-step-2-last-9-chars.png

Or use the Ignore Text (Substrings) parameter, then click the Terms button.

merge-duplicates-linkedin-URL-step-1-ignored-text-terms-button-646w.png

On the Ignored Text tab of the popup, add the common portion of the URL or text string.

merge-duplicates-linkedin-URL-step-1-ignored-text-popup-646w.png

For more help troubleshooting issues with Insycle, refer to our Troubleshooting Issues article.

Frequently Asked Questions

My merge is taking a long time to run. How can I check the progress in Insycle?

When you initiate a merge operation in Insycle, you will see the estimated time to complete. If you notice that the operation is taking longer than expected, there are several factors to consider:

  • If you are using a synthetic merge, this will take longer to run than a native merge.
  • Complex processes, like merges and associations, can take a while. This is because each backend task—from updating associations to changing properties—contributes to the total processing time. For instance, when merging three deals into one, 20-40 operations may be required to update and reparent all related activities from the duplicates into the master.
  • Throughput is measured across all operations; when multiple operations run in parallel, they increase processing load and slow each other down. To optimize processing time, run only one operation at a time.

There are two main ways to monitor the progress of a long-running operation:

Option One: Periodically check the Activity Tracker

The operation will appear in the Activity Tracker once it is complete.

activity-tracker-hubspot-646w.png

Option Two: Periodically refresh the template for the operation

Open the template for the operation. Under Step 1, click the Find button.  As the merge continues, you should see the number of duplicate records gradually decrease in the Record Viewer, which indicates the process is actively working through them.

merge-duplicates-hubspot-companies-check-operation-progress-step-1-template+find.png

Look at the number of results in the Record Viewer.

transform-data-hubspot-contacts-check-operation-progress-record-viewer-1.png

Wait a few minutes and refresh the page. You should see the number of results decrease as Insycle processes the operation.

transform-data-hubspot-contacts-check-operation-progress-record-viewer-2.png

How do I ensure that I am not merging non-duplicate records together?

There are two ways to ensure the records you are merging are indeed duplicates.

First, always run your deduplication templates in Preview Mode before running them in Update Mode. This produces a CSV file showing how your records would have been merged. Then you can ensure that your Merge Duplicates template is working as expected and not merging non-duplicate records together.

Additionally, to ensure a smooth merge, consider narrowing the matching settings in Step 1. Try the Exact Match Comparison Rule instead of Similar Match. Then make sure that you are using actual, uniquely identifying fields—first name, last name, email, and phone number are popular choices. The more tightly defined your filter is, the less likely you are to merge non-duplicate records.

Insycle is having trouble determining a master record. What could be causing this issue?

If the Message column of the CSV report displays this error:

Change rules in Step 3 'Master Selection'. Failed to pick master record because multiple records (X) meet the selection criteria. In 'Master Selection', change, add, or reorder the rules such that only one record matches (if cannot determine master based on field values, use 'Record ID is lowest' as the last rule).

None of the records meet more of the rules than the others do.

There are a couple of things you can try to resolve this:

  1. On the Master tab in Step 3, experiment with reordering or adding additional fields that are likely to have unique values.
  2. At the bottom of the Master tab in Step 3, ensure By Priority is selected, not Absolute.merge-duplicates-step-3-by-priority-match-w-arrow-646w.png
    With By Priority, your master record only has to match one rule. Using Absolute, your master record would have to meet all of the rule criteria. In most cases, it is best to select By Priority.
    If By Priority was used, then none of the records in the duplicate group meet any of the criteria on the list more than the others. In this case, you'll need to experiment, reordering or adding additional rules for fields likely to have unique values.
  3. As a last resort, you can add a rule on the Master tab in Step 3 that says Record ID is lowest, or Create Date is earliest.merge-duplicates-hubspot-contacts-step-3-master-tab-last-resort-rules.png

Can I select which data is retained in my master record on a field-by-field basis?

Yes, Insycle allows you to select which field data to retain in the master record using the Fields tab in Step 3. See the Merge Duplicates module field data retention rules reference for more details.

merge-duplicates-hubspot-contacts-step-3-field-tab-donated-owner-phone-lifecycle-IDs-646w.png

How do I ensure the merged record maintains an active owner?

Currently, neither HubSpot nor Salesforce provides an automated way to prioritize active owners during the merge process. You'd need to verify owner status manually for each merge operation.

However, Insycle's Merge Duplicates module includes an option to prioritize an active owner. 

First, you could add a Master rule under Step 3 to tell Insycle to select the record from each group with an active owner as the master record.

Add a rule with the following parameters:

  • Field: Record owner
  • Condition: active user

merge-duplicates-hubspot-contacts-step-4-record-tab-active-owner-646x247.png

Second, you could create a Field rule to retain the owner who is an active user.

Add a rule with the following parameters:

  • Field: Owner
  • Criteria: From record where value
  • Condition: active user

merge-duplicates-hubspot-contacts-step-4-fields-tab-active-owner.png

Do the field values I use to match need to be exactly the same?

No, matching fields do not need to match exactly. The Similar Match Comparison Rule in Step 1 acts as a fuzzy match, looking for values that may be close but differ by only one character (e.g., a typo), and broadens the search.

step-1-email-only.png

This search behaves like when Google shows results for a slightly different term or says, “Did you mean...” For example, if an Email of “huey@coahulldu.co” is found, it could include records with the values “hueyy@coahulldu.co" or "hue.y@coahulldu.co” as a match.

step-2-group-w-similar-match.png

You should be careful when using Similar Match as the looser criteria can incorrectly identify non-duplicates as duplicates. 

Review the Understanding Similar Matching best practices for more details.

How do I preserve data from a read-only field during a merge?

By default, merged records retain the value from the master record. If you want to keep a specific value from a read-only field that's not in the master, you have two options:

1. Copy to a writable custom property: In your CRM, create a custom field to copy the read-only field value into. Then run a Copy operation in the Transform Data module before performing the merge. Finally, set up Merge Duplicates rules on the Fields tab of Step 3 to retain the value of that custom property that meets your criteria.

transform-data-contacts-step-2-copy-email-to-backup-646w.png

2. Use master selection rules: Set the master selection rules on the Master tab of Step 3 to retain the record with the read-only value you want to keep. This guarantees the correct record becomes the master without trying to change the read-only property itself.

merge-duplicates-hubspot-contacts-step-3-master-tab-email-646w.png

I have a list of duplicates I need to merge. Can I deduplicate them using Insycle?

Yes. You can use an existing list of duplicates with Insycle to deduplicate it in bulk, following these steps:

  1. Prepare a CSV file with columns for the record IDs and a "Merge Master" column. In the "Merge Master" column, mark which record should be kept after merging.
  2. Create a custom field called "Merge Master" in your CRM.
  3. Use the Magical Import module to import your CSV file into the CRM, updating the "Merge Master" field for the relevant records.
  4. Go to the Merge Duplicates module and set up a filter to select records based on the "Merge Master" field.

Learn more about deduplicate records using a CSV.

How many duplicates can I merge into one master record?

You can merge up to 100 duplicate records into a single master record.

By default, Insycle skips any duplicate group containing more than 5 records. If you have duplicate groups with more than 5 records, you may want to adjust the Exclusions setting at the bottom of the Master tab in Step 3. The Skip duplicate groups with more than X records per group setting can be increased up to 100 records per group to ensure all duplicates in large groups are processed.

This default limit is a precaution to prevent duplicate matching criteria that are too broad in Step 1 from accidentally merging many non-duplicate records. If you plan to set this number higher, it is a good idea to run Preview Mode first to ensure your deduplication template is operating as you intend.

merge-duplicates-hubspot-contacts-step-3-master-tab-exclusions-w-arrow-646w.png

Additional Resources

Related Help Articles

Related Blog Posts