Customize Bulk Deduplication Using Exclusions and Pre-Defined Masters

Navigate to Data Management > Merge Duplicates, and select the database and record type in the top menu. Then explore the existing templates for an existing solution that may be close to what you need.

Next, go through Steps 1-5 of the module to create a Preview CSV. Make sure to pause there; you don't want to Update your records yet. Follow the instructions found in the Bulk Merge Duplicate People and Companies article through to Step 5: Preview Merged Changes in CSV Report.

Insycle will generate a preview CSV and send it to your email.

Open the Preview CSV file from your email in a spreadsheet application.

The CSV will have a row per record and include the following columns:

• The Result of the action
• A Message with details
• The Duplicate Group ID which indicates which records will be merged together
• A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
• All fields used to identify the duplicates in Step 1
• The record ID, record name or email, and Deeplink to the CRM record
• All fields selected for the Master and Fields rules in Step 4. Note that if a field is used on both tabs, it will only appear once in the CSV
• Any additional fields added to the Layout tab in Step 2

Add two new columns to the CSV:

• Deduplication Exclude
  and
• Deduplication Master

Now, given the context from all the fields available in the report, update the CSV as needed:

• Enter TRUE in the Deduplication Exclude column for any row you'd like to exclude from the merge.
• Enter TRUE in the Deduplication Master for the row you'd like to designate as the master from within a duplicate group.

When you're done filling in the Deduplication Exclude and Master columns, remove all columns from the CSV except for:

• Record ID
• Deduplication Exclude
• Deduplication Master

When finished, save the CSV as a new file to your computer.

In your CRM, create two custom fields:

• Deduplication Exclude
  and
• Deduplication Master

For the field type, pick "Boolean" or "Checkbox."

The underlying field names for these fields in the CRM are: 

• deduplication_exclude
• deduplication_master

After creating the CRM fields, refresh the data in Insycle by logging out and then logging back in. You could also navigate to Settings > Sync Status, and next to the account name, click the Sync changes from last day button (lightning bolt icon).

Open the Magical Import module and load the saved CSV file with your new custom fields.

Under Step 1, map the CRM ID column to the CSV Record ID field. The other two fields will map automatically.

Under Step 2, set the ID field for the Matching Criteria field and select the Records Mode, "Only update existing," because those records already exist in the CRM. You're just "tagging" those existing records for deduplication.

Then click the Import button under Step 5 to import the CSV.

Now that records have the deduplication tags in the CRM, you can use the custom fields to tailor the cleanup process.

Return to the Merge Duplicates module. In Step 1, click the Filter button to exclude records from deduplication.

For the filter settings in the popup, select the Deduplication Exclude field, the "not any of" Condition, and enter "Yes" in the value field.

To designate your chosen records as the masters, in Step 4, remove all rules and add just one rule: Deduplication Master, "is," "Yes."

Preview Your Changes in a New CSV Report, Then Apply the Updates to Your CRM

Under Step 5, use Preview mode to confirm that the deduplication works as expected with all the customizations. 

If everything in the CSV looks correct, return to Insycle and move forward with applying the changes to your CRM using Update mode.

Yes. You can merge specific records using a CSV file containing the records you want to combine. Here's how:

• Prepare a CSV file with columns for the record IDs and a "Merge Master" column. In the "Merge Master" column, mark which record should be kept after merging.
• Create a custom field called "Merge Master" in your CRM.
• Use the Magical Import module to import your CSV file into the CRM, updating the "Merge Master" field for the relevant records.
• Go to the Merge Duplicates module and set up a filter to select records based on the "Merge Master" field.

See the Step-by-Step Instructions above for full details.

No, your field data does not need to match exactly. The Similar Match Comparison Rule found in Step 1 looks for values that may be close but with a one-character difference (maybe a typo) and broadens the search.

This search behaves like when Google shows results for a slightly different term, or says “Did you mean...” For example, if an Email of, “huey@coahulldu.co” is found, it could include records with the values “hueyy@coahulldu.co," or "hue.y@coahulldu.co,” as a match.

When you do use Similar Match, make sure to carefully review the results to ensure the duplicates being identified are what you're expecting.

Review the Understanding Similar Matching best practices for more detail.

Insycle shows 50 records on the module screen as a preview, this isn't the entire list of records. View the Preview CSV report to see the results for all records.

Insycle can process thousands of duplicate groups in one operation. Potentially, you could deduplicate your entire database in one operation. 

You can merge up to 100 duplicates into one single master record. 

If you have duplicate groups that contain more than five records, you may want to change the value in Skip duplicate groups with more than 5 records per group under Step 3 to make sure you can get them all.

This is a precaution to ensure that if you use a duplicate matching filter that is too broad in Step 1, you don't accidentally merge many non-duplicate records. If you're going to set this number at a high level, it's a good idea to run Preview Mode first to ensure your deduplication template is operating as intended.

All plans include unlimited usage, unlimited users, and unlimited operations. During the free trial, there is a cap of 500 records updated, cleansed, or merged. See the pricing page for more details.