Salesforce Merge Duplicates Overview

Insycle supports the following Salesforce record types:

• Contacts
• Accounts
• Leads
• Opportunities
• Custom record types

You can select the record type you would like to import at the top of the module.

Step 1: Configure Rules to Identify Duplicates

In the Data Management > Merge Duplicates module, pick the record type and explore the default templates for an existing configuration close to what you need.

Each row in Step 1 is for a field you want to look at for duplicates, along with some parameters on what to look for. You want to choose fields that, in combination, give a high degree of certainty that the matched records are duplicate records. See the Bulk Merge Duplicate People, Companies article for more details.

If you'd like to look at the data in two different fields (that contain similar data) as if it were one, you can set up Related Fields under the Advanced tab.

The Conditions tab provides rules that one or more of the records in a duplicate group must meet. These options let you choose fields that are required, can be empty, or specify values that must be included.

Step 2: Analyze the Identified Duplicates

When two or more records represent the same entity (person, company, or other) based on your matching rules, they are clustered together into duplicate groups. Each duplicate group shows the total number of records that were identified as duplicates based on your settings. For example, if you had three records for the same person, it would count as one duplicate group with three duplicate records.

Step 3: Choose Whether to Merge in Bulk or Manually

The most efficient and sustainable way to merge duplicates is in Bulk mode. This allows you to set rules for determining the master record automatically across all records in your database. You'll be able to use saved templates and recipes to repeat the process on a regular basis.

In Manual mode, you have complete control over which records are merged together by selecting them from the Record Viewer. Manual mode should be reserved only for cases where you need a careful, controlled process. Learn more about merging duplicates in Manual mode.

Step 4: Set Rules for Master Record Selection and Data Retention

After selecting Bulk mode in Step 3, you need to define how all of the matching duplicate groups should be merged at scale in Step 4.

Configure Rules to Automatically Select the Master Record

First, select the matching method—Priority Match or Absolute Match. Most de-duplication operations should use Priority Match. Learn more about these options in the Bulk Merge Duplicate People, Companies article.

On the Records tab of Step 4, you define how the duplicate groups should be merged at scale by creating rules that tell Insycle how to select the record from each group to become the master. The master is the record that will remain after the merge.

For example, if you had three records representing the same contact, they would all be merged into the one master record. The other two records would not exist anymore.

Configure Rules That Determine Values to Keep

Duplicates may be exact match versions of another record, but often there is only partial data overlap between them. When data is split between two records, both may contain unique and important information you want to keep.

Under Step 4, click the Fields tab. By default, Insycle will keep the most recent values.

For each field you want to control the data retention for, you need to tell Insycle which record the data should be taken from. This is merged into the master.  Any other data that is not in the master or not copied to the master is removed.

Learn more about configuring data retention and master record selection.

Preview Merged Changes in CSV Report

Now, with the filters and master record set up, you can preview the changes you are making to your data in a CSV. That way, you can check to ensure your deduplication configuration is working as expected before those changes are pushed to your live database. 

Under Step 5, click the Review button and select Preview mode.

On the Notify tab, select recipients and add context to the report email.

On the When tab, click the Run Now tab, and select which records to apply the change to (you could do All, but if you have a large number of records, you may just want to do a chunk for your preview), then click the Run Now button.

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application.

The Duplicate Group ID indicates which records will be merged together. 

The Status column indicates:

• Duplicate – The record is part of a duplicate group.
• Master – The master record chosen for the duplicate group based on default behavior and your Record rules. Review the selections in this row to determine whether the appropriate records are being chosen.
• Master (After) – This appears only if at least one or more fields have been specified in Step 4 on the Fields tab. For each duplicate group, the Master (After) row shows the values the final record will contain based on your Field rules and the default behavior. 
• Error – If Insycle is not able to determine which record would be the master, an error message will appear here. See the Troubleshooting section below for more details.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your Salesforce Records

When you're satisfied with the results in your preview, you can apply the merge changes to Salesforce.

Under Step 5, click the Review button again, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to Salesforce. If you have a large number of records, you may want to do a smaller batch to review the results in Salesforce.

After you've seen the results in Salesforce and you are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a Recipe.

Return to the Template menu at the top of the page and click copy to save your configurations as a new version of the template you started with. Then click the pencil to edit your new template name.

Under Step 5, click the Review button, and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (hitting Enter after each address) and provide context in the message subject or body.

On the When tab, select Automate, and configure the frequency you'd like the template to run. When finished, click Schedule.

You can view all your scheduled automations at any time on the Operations > Automations page.

Learn More:

• Automation and Manual Operations
• Recipes

With the Activity Tracker, you have a complete audit trail and history of changes made through Insycle, including processes run in Preview mode or data syncs. At any time you can download a CSV report that lets you see all of the changes that were made in a given run of the operation.

Navigate to Operations > Activity Tracker, search by module, app, or template name, then click the Run ID for the operation.

If you're running into issues because the same person exists in Salesforce as both a lead and a contact, you can use the Merge Duplicates module to deduplicate leads and contacts together.

Open the Merge Duplicates module and pick the Contacts record type.

Then, under Step 1, check the Include Leads checkbox.

Learn more about merging duplicate individuals looking at both leads and contacts.

Having your Salesforce and HubSpot CRMs set up to sync can make cleaning up duplicates tricky. You need to determine the appropriate “master record” to use across both CRMs and consider the merging process. Often, your settings in each platform impact how the merge takes place.

When you deduplicate accounts in Salesforce, the master is kept in sync with the original HubSpot record, indicated by a Salesforce Account ID value. However, the deduplication only takes place on Salesforce, leaving duplicate companies in HubSpot. Since HubSpot doesn't allow you to deduplicate companies while the sync is active from within the HubSpot app, you need another option.

Insycle allows you to merge duplicate HubSpot companies and Salesforce accounts while keeping things simple and your sync intact.

To learn more, see Deduplicate HubSpot Companies and Salesforce Accounts. 

For situations where there are no common rules you can apply for identifying duplicates for all or some of the records, you may need more granular control for picking records to include or exclude from the process.

To do this in bulk, you can create a CSV and use the Magical Import module.

In these cases, you can use CSV files to customize your bulk merging, designate master records, and exclude records from deduplication. Then you can import the CSV from the Magical Import, and use the Merge Duplicates module for complete control over the final merge operation. Learn how to customize merging Duplicates in bulk using a CSV.

To do this one record at a time, you can use Manual mode of the Merge Duplicates module.

In Manual mode, you have complete control over which records are merged together by selecting them from the Record Viewer. Manual mode should be reserved only for cases needing a careful, controlled process. Learn more about merging duplicates in Manual mode.

Insycle uses Salesforce's underlying APEX merge API, which supports leads, contacts and accounts. For more information, see Salesforce's Apex Developer Guide.

When merging opportunities and any other standard or custom object, Insycle performs a synthetic merge:

• Fields (i.e., phone number): Data from the master record is kept. When a field value is empty in the master, it picks a non-empty value from the most recently updated duplicate automatically. When in doubt about conflicting field values, include those fields in the CSV report by adding them to the Master Selection section. Their values will also show on the audit trail.
• Relationships: Insycle inspects the schema metadata for relationships to the duplicate records and relinks those relationships to point to the master record instead of the duplicate. For example, that's how it would re-link "Notes" in the duplicate records into the master record.

If you have enabled Salesforce's setting for relating a contact to multiple accounts, Insycle's Merge Duplicate module ensures that all account relationships are preserved during the merge process. It automatically reassigns both primary (direct) and secondary (indirect) account links to the master record while removing any redundant relationships. This provides a comprehensive view of the contact's interactions across all linked accounts within your business.

Insycle intelligently handles scenarios where duplicate contacts are linked to the same secondary accounts. It identifies these overlapping relationships and automatically removes the extraneous links from the records being merged, while retaining the links on the master record. This prevents the common Salesforce error: "Can't merge contacts. These contacts have the same related account. Remove the redundant account-contact relationships and then try merging again."

By addressing these relationship mapping challenges, Insycle streamlines the deduplication process, saves significant time and effort, and ensures valuable account-contact relationship data is not lost due to merging duplicates with multiple links.

Learn more about deduplicating with Salesforce’s feature for linking contacts to multiple accounts.

When setting up your merge operation in the Merge Duplicates module, add extra fields to be included in the CSV report so you have the data later for undoing changes and/or just for general review.

Under Step 4 on the Record tab, add any key fields to the bottom of the list using the "exists" Condition.

Before running a large merge operation, you can export all records directly from Salesforce to a CSV file. This can preserve all the record details, which can help later for undo and/or just for general review.

Create a Targeted Salesforce Report

To select only the necessary fields and filter out inapplicable records, you can create a Salesforce Report. This gives you a targeted export, omitting unnecessary data. Once configured and saved, you can reuse this report before each bulk merge operation.

In Salesforce, if the Reports tab doesn't appear at the top, click the App Launcher icon (nine squares) in the top left corner and search for "Reports."

Create a new report based on the specific data type, and add all the desired fields. Add filters to narrow the results. Save and Run the report.

On the results page, click the arrow next to the Edit button and select Export. In the Export popup, select Details and CSV as the format.

Export All Data for a Record Type

To get all field data for all records of the same type, you can use Salesforce's Data Export Service. There are no filter/segmenting options, so if you have a large dataset, this can take a while to run.

In Salesforce, navigate to Setup > Data > Data Export.

Click Export Now, check the box for the record type, then click Start Export. You'll receive an email when the file is ready for download. 

You have duplicate records that have been identified by Insycle but not all of them are merging into the master. Check to see how many duplicates are in the affected duplicate groups. If you have duplicate groups that contain more than five records, you may want to change the value in Skip duplicate groups with more than 5 records per group to make sure you can get them all.

This setting is intended to protect against the accidental merging of non-duplicate records if the filter in Step 1 is too broad.

If the Message column of the CSV report displays this text:

Change rules in Step 4 'Master Selection'. Failed to pick master record because multiple records (X) meet the selection criteria. In 'Master Selection', change, add, or reorder the rules such that only one record matches (if cannot determine master based on field values, use 'Record ID is lowest' as the last rule).

This means that based on all the rules, Insycle could not figure out which record in the duplicate group would be the master. None of the records meet more of the rules than others.

There are a few things you can try to resolve this:

1. Under Step 4, on the Record tab, experiment with reordering or adding additional fields that are likely to have unique values.
2. In the Step 4 heading, check to ensure that you have Priority Match selected and not Absolute Match.

   
   With Priority Match, your master record only has to match one rule. Using Absolute Match, your master record would have to meet all of the rule criteria. The majority of the time, it is best to select Priority Match.

   If Priority Match was used, then none of the records in the duplicate group meet any of the criteria on the list more than the others. In this case, you'll need to experiment with the Record tab, reordering or adding additional rules for fields likely to have unique values.

3. As a last resort, you can add a rule on the Record tab of Step 4 that says Record ID is lowest, or Create Date is earliest.
   

There are a couple of things to look at that may be misidentifying records as duplicates.

First, you may need a better unique identifier. Under Step 1, if you only use fields that could correctly contain the same values in multiple records, these aren't unique identifiers. In this case, you are likely to identify unrelated records as duplicates and may accidentally merge them.

Unique identifiers are data that is unlikely to be shared by any other record unless it represents the same underlying entity. Fields that are commonly used in deduplication include phone numbers, email, mailing addresses, or ID numbers.

Second, this may indicate the Comparison Rule under Step 1 is too broad. Try using the Exact Match comparison rule instead of Similar Match. Similar Match looks for values that may be close but with a one-character difference (maybe a typo) which broadens the search. 

Remember, always run your deduplication in Preview Mode to confirm things are working as expected before running them in Update Mode and applying the changes to your CRM records.

Most of the time, when Insycle can't find duplicates, it is due to your matching rules in Step 1. It is important to analyze the underlying data to better understand how to set up your rules. A useful exercise can be to set up your matching filters to look for exact matches of just First Name and Last Name.

When you click Find, these rules can show you a broad overview of what duplicates are potentially in your database and what fields might be useful to include in your matching fields. These settings are just for discovery and should not be used for a final merge operation; many people can have the same first and last names and are not duplicates. 

To get further context, on Step 2, click the layout gear button on the right side of the title bar. Here, you can add any field in your database as a column to the duplicate group review to better understand the data inside these records.

It can take a while for Insycle to find and match duplicates if the fields being used to identify them have very long values. The longer the values, the longer it takes Insycle to process the data and generate the results. This might come up when looking for matches based on long ID numbers, LinkedIn bio links, or other URLs with long strings attached (ex, https://www.linkedin.com/in/svadin%C3%ADr-n%C4%9Bmec-1234b31a3/).

You can speed this up by limiting how much of the value Insycle looks at.

If the beginning or ending portion of the values are all unique, you can limit the comparison to the first or last several characters using the Match Parts parameter under Step 1. 

Or use the Ignore Text (Substrings) parameter, then click the Terms button.

On the Ignored Text tab of the popup, add the common portion of the URL or text string.

Occasionally, after you run a merge operation, you may see the message: "Underlying error message from Salesforce" in the Merge Result popup. This indicates there is an issue on the Salesforce side that needs to be investigated in Salesforce.

It's worth checking for any rules relating to the record type in the Duplicate or Matching Management settings for your Salesforce account.

Go to Setup > Data > Duplicate Management > Duplicate Rules or Matching Rules.

You may need to turn these rules off before Insycle can proceed with the merge to prevent the error. You could completely disable them if you want to manage all duplicates through Insycle or disable and enable the rules as needed.

If you are running a bulk merge operation on custom objects for the first time, you may see the message "Please contact support@insycle.com to enable bulk merge for type: [Custom Object Name]."

By default, you can manually merge custom objects that have been synced with Insycle, but before you can bulk merge duplicate custom objects, they must be enabled for your account. This message indicates the step has not been completed yet. 

To have bulk custom object merging enabled for your account, please contact Insycle support by email, or using the live chat that appears at the bottom of every page on the Insycle website or the app.

Learn more about using Salesforce custom objects and fields in Insycle.

Currently, Salesforce doesn't have an automated way to prioritize active owners during the merge process. You'll need to verify owner status manually for each merge operation.

When merging records in Insycle, you can create a Field rule under Step 4 to keep the owner who is an active user.

Add a rule with the following parameters:

• Field: Owner
• Criteria: From record where value
• Condition: active user

When using two or more fields to identify duplicates, records can still be considered matches even if one of the field values is blank. You just need to specify which field(s) allow a blank value.

Under Step 1, configure your matching rules in the Simple tab, then click the Conditions tab.

All the matching fields you included will automatically appear with the Value Required in All Records condition selected. Change the condition to Empty Allowed in Any Record to allow empty values for certain fields. You can also use the At Least One Record with Non-Empty condition to help you determine which is the master record. Make sure at least one field remains required and is a reliable unique identifier to ensure the records are really duplicates.

For example, on the Simple tab, you may have the matching fields: First Name, Last Name, and Phone Number. But on some of your records, the Phone Number field may be empty. Using the Empty Allowed in Any Record or At Least One Record with Non-Empty, all records with the same name, same phone number, and no phone number will be considered duplicates.

Yes. This can be done, for example, if you want to look at both the Phone Number field values and Mobile Phone Number field values as a single pool of values to compare between records and identify duplicates.

Using the Related Fields feature, you can use two different fields (that contain similar data) as matching fields to catch more duplicates. You can set up Related Fields in the Advanced tab.

There are two ways to make sure that the records that you are merging are indeed duplicate records.

First, always run your deduplication templates in Preview Mode before running them in Update Mode. This produces a CSV that shows you how your records would have been merged. Then you can ensure that your Merge Duplicates template is working as expected and not merging non-duplicate records together.

Additionally, to ensure a smooth merge process, consider narrowing down the matching settings in Step 1. Try the Exact Match Comparison Rule instead of Similar Match. Then make sure that you are using actual uniquely identifying fields—first name, last name, email, and phone number are popular choices. The more tightly defined your filter is, the less likely you are to merge non-duplicate records.

If the Message column of the CSV report displays this text:

Cannot determine master record because multiple records (#) satisfy the master selection rules. In ‘Master Selection’, change/add/reorder the rules such that only one record satisfies them (if cannot determine master based on field values, use ‘ID is lowest’ as the last rule).

This means that based on all the rules, Insycle could not figure out which record in the duplicate group would be the master. None of the records meet more of the rules than others.

There are a couple of things you can try to resolve this:

1. Under Step 4, experiment with reordering or adding additional fields that are likely to have unique values.
2. In the Step 4 heading, check to ensure that you have Priority Match selected and not Absolute Match.
   
   With Priority Match, your master record only has to match one rule. Using Absolute Match, your master record would have to meet all of the rule criteria. The majority of the time it is best to select Priority Match.
   If Priority Match was used, then none of the records in the duplicate group meet any of the criteria on the list more than the others. In this case, you'll need to experiment, reordering or adding additional rules for fields likely to have unique values.
3. As a last resort, you can add a rule on the Record tab of Step 4 that says Record ID is lowest, or Create Date is earliest.
   

Yes. You can merge specific records using a CSV file containing the records you want to combine. Here's how:

• Prepare a CSV file with columns for the record IDs and a "Merge Master" column. In the "Merge Master" column, mark which record should be kept after merging.
• Create a custom field called "Merge Master" in your CRM.
• Use the Magical Import module to import your CSV file into the CRM, updating the "Merge Master" field for the relevant records.
• Go to the Merge Duplicates module and set up a filter to select records based on the "Merge Master" field.

Learn more about customizing bulk deduplication from a CSV.

Yes, Insycle allows you to control data retention in the master record using the Fields tab under Step 4. See the Bulk Merge Duplicate People, Companies article for more details.

Yes. You can exclude records from deduplication by creating a CSV with a "Deduplication Exclude" field.

First you'll export a Preview CSV from the Merge Duplicates module, add an exclude column, and specify which records should be excluded from the merge process. Next, create a custom field in your CRM to facilitate the merging. Use the Magical Import module to import the edited CSV file into the CRM, populating the new custom field. Finally, utilize this custom field to merge the remaining duplicate records in the Merge Duplicates module.

Learn how to customize bulk deduplication using exclusions.

Yes, there are several ways to share details and get approval before merging duplicates.

You can manually approve master records and mark them in a CSV, then use Insycle to bulk deduplicate down to those master records. See the Customize Bulk Deduplication Using Exclusions and Pre-Defined Masters article to learn more.

Or, you can run the Merge Duplicates module in Preview Mode and then deliver the preview CSV that Insycle generates. The CSV report includes your entire merge operation down to individual duplicate groups but does not update your live data. Then your team can approve the merge based on this report, before running Merge Duplicates in Update Mode.

Additionally, team members can review duplicates and manually select the master for each record under Step 4. Review the Manually Merge Duplicates article for more detail.

No. The Similar Match Comparison Rule found in Step 1 looks for values that may be close but with a one-character difference (maybe a typo) and broadens the search.

This search behaves like when Google shows results for a slightly different term or says, “Did you mean...” For example, if an Email of “huey@coahulldu.co” is found, it could include records with the values “hueyy@coahulldu.co," or "hue.y@coahulldu.co,” as a match.

Do pay close attention when using Similar Match as the looser criteria can incorrectly identify non-duplicates as duplicates. 

Review the Understanding Similar Matching best practices for more detail.

Yes, Insycle can analyze leads and contacts together and deduplicate across those record types. See the Deduplicate Across Salesforce Leads and Contacts article to learn more.

Yes, Insycle solves numerous deduplication-related issues when Salesforce and HubSpot are syncing. See the Deduplicate Salesforce and HubSpot While Keeping the Sync Active article to learn more.