Pipedrive Merge Duplicates Overview

Insycle supports merging the following Pipedrive record types:

• Contacts
• Organizations
• Deals

You can select the record type that you would like to deduplicate at the top of the module.

When merging duplicates, Insycle goes through the following steps for all Pipedrive record types:

1. Select the master record according to the rules set in Step 3 on the Master tab.
2. Determine field values to retain in the master record based on rules configured in Step 3 on the Fields tab.
3. For fields without specific rules, identify non-empty values using a "fill in the blanks" method:
   • Check whether the master record has any empty fields.
   • If empty, copy values from the most recently updated record in the duplicate group.
4. Update the master record by:
   • Applying the field values chosen in step 2 (from Field rules).
   • Adding the non-empty values identified in step 3 (using "fill in the blanks").
   • Preserving all other existing values in the master record.
5. Merge the duplicate records and reassign all related objects to the master record.
6. Store an audit trail report accessible from the Activity Tracker and send an email notification to the designated recipients.

If you're uncertain about how conflicting field values will merge, you can add those fields to the Preview CSV report by including them in the Layout via the gear icon in Step 2.

This is also useful if there are fields you're worried about backing up; including them in the CSV report allows you to refer back to the values in the CSV sent to your email or downloaded from the Activity Tracker.

Navigate to Data Management > Merge Duplicates, pick the record type, and explore the default templates for a pre-built solution.

Each row in Step 1 is for a field you want to look at for duplicates, along with some parameters on what to look for. You want to choose fields that, in combination, give a high degree of certainty that the matched records are duplicate records. See the Setting the Criteria for Finding Duplicate Matches article for more details on configuration options.

Step 2 displays all duplicate groups identified in Step 1, allowing you to review and verify the matches before proceeding. Records that match your Step 1 criteria are automatically clustered into duplicate groups — each group represents a single entity and shows how many duplicate records it contains. For example, three records for the same person would appear as a single duplicate group containing three records.

Click the chevron  at the right end of a duplicate group row to expand it and examine the individual records within that group. You can also add fields as columns to provide more context when evaluating the matches by clicking the gear icon.

To exclude a duplicate group from all future deduplication analysis, click the X on the group row. Excluded groups will not appear as duplicates or be included in merges, regardless of the duplicate detection template used. To review and manage excluded groups, click the Exclusions button in the Step 2 header.

Step 3 is where you configure how duplicate groups are consolidated—from choosing your operation mode to defining master selection rules and field retention logic. This step combines operation selection with the merge logic configuration through multiple tabs.

At the top of Step 3, choose between Bulk and Manual modes in the header:

• Bulk mode (recommended) automatically merges all duplicate groups according to the master record selection rules you define in the Master tab. This is the right choice for most deduplication processes, and the only mode that supports templates, Recipes, and automation.
• Manual mode lets you review and select specific records to merge individually from the Record Viewer. Reserve this for high-value records or complex scenarios where you want hands-on control over each merge. Learn more about merging duplicates in Manual mode.

Next, define how all matching duplicate groups should be merged at scale.

Configure Rules to Automatically Select the Master Record

On the Master tab, you define how the duplicate groups should be merged at scale by creating rules that tell Insycle how to select the record from each group to become the master. The master is the record that will remain after the merge.

For example, if you had four records representing the same contact, they would form a duplicate group of four records, which would be merged into a single master record. The other three records no longer exist.

Here, you also select the processing method—By Priority or Absolute. Most de-duplication operations should use By Priority. Learn more about these options in the Choosing Which Records to Merge Using a CSV article.

Configure Rules That Determine Values to Keep

Duplicates may be exact match versions of another record, but often, there is only partial data overlap between them. When data is split between two different records, both may contain unique and important information you'd like to keep.

Under Step 3, click the Fields tab. For each field you want to control the data retention for, select the field and specify where the data for that field should be taken from. This is merged into the master. Any data that is not in the master or not copied to the master is removed.

See Keeping the Right Data When Records Merge for more details.

Preview Merged Changes in CSV Report

Now with the filters and master record set up, you can preview the changes you are making to your data. That way, you can check to ensure your deduplication configuration is working as expected before those changes are pushed to your live database. The CSV report that Insycle generates includes records from all the duplicate groups.

Under Step 4, click the Review button and select Preview mode.

Click the Next button to go to the Notify screen, where you can select recipients for the email report and add additional context.

On the When tab, click the Run Now tab, select which records to apply the change to (in most cases, this will be All), and then click the Run Now button.

Insycle will generate a preview CSV and send it to your email. Open the CSV file from your email in a spreadsheet application and review the data.

There's a row for each record, which includes:

• The Result of the action
• A Message with details
• The Duplicate Group ID which indicates which records will be merged together
• A record Status that identifies which were picked as master and which were identified as duplicates and merged into the master. See below for more details
• All fields used to identify the duplicates in Step 1
• The record ID, record name or email, and Deeplink to the CRM record
• All fields selected for the Master and Fields rules in Step 3. Note that if a field is used on both tabs, it will only appear once in the CSV.
• Any additional fields added to the Layout tab in Step 2

 The Status column indicates:

• Duplicate – The record is part of a duplicate group.
• Master – The master record is chosen for the duplicate group based on default behavior and your Master rules. Review the selections in this row to determine whether the appropriate records are being chosen.
• Master (After) – For each duplicate group, a special row –Master (After)– will appear in the CSV report. The Master (After) row shows the values the final record will contain based on your Field rules and the default behavior.
• Error – If Insycle is not able to determine which record would be the master, an error message will appear here. See the Merge Duplicates Troubleshooting article for next steps.

If everything looks good, return to Insycle and move forward with applying the changes.

Apply Changes to Your CRM Records

When you're satisfied with the results in your preview, you can apply the merge changes to your CRM.

Under Step 4, click the Review button again, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to the CRM. If you have a large number of records, you may want to do a smaller batch to review the results in your CRM.

After you've seen the results in the CRM and you are satisfied with how the operation runs, you can save your configuration as a template and set up automation so this merge operation runs on a set schedule. If you have several templates you'd like to run together automatically, you can create a Recipe. 

Return to the Template menu at the top of the page and click Copy to save your configurations as a new version of whatever template you started with. Then click the pencil to edit your new template name.

Under Step 4, click the Review button, and select Update mode.

On the Notify tab, select the send option appropriate for your automation: Always send, Send when errors, or Do not email.

Add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can also provide additional context in the message subject or body.

On the When tab, select Automate, and configure the frequency you'd like the template to run. When finished, click Schedule.

You can view all your scheduled automations at any time on the Operations > Automations page.

With the Activity Tracker, you have a complete audit trail and history of changes made through Insycle, including processes run in Preview mode or data syncs. At any time, you can download a CSV report that shows all the changes made in a given operation run.

Navigate to Operations > Activity Tracker, search by module, app, or template name, then click the Run ID for the operation.

If you've configured the Insycle Run ID property in Pipedrive, you can cross-reference these operations in your CRM records.