How to Format Fields in Bulk and Automatically to Ensure Data Consistency

Data within an organization often originates from multiple sources, including website forms, internal data entry, integrations, and APIs. This can make it challenging to enforce consistent formatting across specific fields, complicating efforts to segment and search for records effectively.

With Insycle, you can format any field in your database using pre-defined rulesets and automate those templates to ensure your fields are consistently formatted correctly.  

Existing data in any field can be formatted in bulk using the Transform Data module. Alternatively, if you have an export from a third-party application or other data in a CSV format, the new data can be formatted on import using the Magical Import module.

Format in Bulk Using the Transform Data Module

The Transform Data module is ideal for formatting fields like names, phone numbers, and addresses. You can easily format and standardize any free-text field in your CRM in bulk.

Process Summary

  1. Find and review records for formatting.
  2. Set up functions to format field values.
  3. Preview the changes.
  4. Apply changes to the CRM records.

 

Step-by-Step Instructions

Step 1: Filter and Review Records to Be Formatted

Navigate to Data Management > Transform Data, pick a record type, and explore the default templates for a pre-built solution.

Under Step 1, adjust the settings to filter the CRM records down to those you would like to format. This ensures you aren't trying to format fields that don't contain any data. And it is best practice to only tackle one field at a time.

The example below will search for contacts with the first name in all caps:

  • Field: First Name 
  • Condition: regex
  • Value: [A-Z]{3,}

This regular expression is looking for:

  • Uppercase letters from A to Z: [A-Z]
  • Three or more occurrences of these uppercase letters: {3,}
transform-data-hubspot-contacts-step-1-first-name-all-caps.png

You can add any filter relevant to your use case—lifecycle stages, record creation dates, engagement triggers, etc. Insycle can use any data in your CRM in the filter step.

Click the Search button and scroll down to the Record Viewer at the bottom of the screen. All records that match your filter will appear. You can alter the fields that appear in this preview from the Layout tab in Step 1.

format2.png
Step 2: Configure Formatting to Apply to Fields

Under Step 2, give Insycle instructions on what formatting and cleanup changes to make to the identified records.

Select the Field that contains the value you want to start with, then select the Function and enter the parameters. Click the plus at the end of the row to add an additional function for this field.

Each function will use the value output from the previous function, so their sequence matters.

There are pre-built functions for standardizing phone numbers, countries, states, domains, and other values, so you should explore the options in the Function dropdown or review the Function Catalog.

format3.png

In the above example, these standardizations are being made to the First Name in each contact record:

  • Remove whitespace from before or after the value.
  • Remove salutations such as Mr or Mr. or Mrs or Mrs. 
  • Capitalize the first name value.

The next example will format specific parts of the Street Address field values. The "Map: Terms" Function will look for the Existing Text and if found, will change it to the New Text value. Here in the Existing Text fields, the bar character "|" between the values means OR, as in, look for "street" or "st.".

transform-data-hubspot-contacts-standardize-street-address-step-2.png

Altogether, these rules will standardize the Street Address like this:

  • Street → St
  • Drive → Dr
  • Avenue → Ave
  • Boulevard → Blvd
Step 3: Preview Changes and Update CRM Records

Preview Changes in the CSV Report

With the filters and functions set up, you can preview the changes. It's important to verify that your formatting is working as expected before those changes are pushed to your live database.

Under Step 3, click the Review button, then select the Preview option.

format5.png

On the Notify tab, add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can add additional context to the subject line and email body.

On the When tab, click Run Now and select which records to apply the change to (you could do All, but if you have a large number of records, you may just want to do a chunk for your preview), then click the Run Now button.

Open the CSV file from your email in a spreadsheet application and review the values for each row.

The Result column indicates the outcome of the operation for each record. The possible results are:

  • Updated - The transform was applied successfully.
  • Failed - The transform could not be applied. See the Message column for details.
  • Unmodified - The transform was not applicable to the record.

To the far right, you can see the (Before) and (After) values side-by-side; this example shows First Name (Before) and First Name (After).

If the results don't look as you expected, go back to your filters in Step 1 and functions in Step 2, and try making some adjustments. Then, preview again.

transform-data-hubspot-contacts-faild-unmodified-result-csv-646w.png

Apply Changes to the CRM

If everything in your CSV preview looks correct, return to Insycle and proceed with applying the changes to the live CRM data.

Under Step 3, click the Review button again, and this time select Update mode.

On the When tab, you should use Run Now the first time you apply these changes to the CRM. If you have a large number of records, you may want to do a smaller batch to review the results in your CRM.

format7.png

After you've seen the results in the CRM and you are satisfied with how the operation runs, you can save all of the configurations as a template, and set up automation so this formatting operation runs on a set schedule.

If you have several templates you'd like to run together automatically, you can create a Recipe. Additionally, HubSpot users can integrate Insycle Recipes into HubSpot Workflows.

By automating with a template, you'll ensure that your fields are consistently and automatically formatted on an ongoing basis. 

Pro Tip: Complex Formatting Example

You can combine functions to isolate parts of a value, and apply formatting before copying to a destination field.

Extract First and Last Name from email address

You can look for email addresses that follow the "firstname.lastname@domain.com" format, then combine functions to parse and extract potential first and last name values from email addresses, format them properly with capitalization, and populate the respective First Name and Last Name fields accordingly.

In this example, the filter under Step 1 is set up to:

  • Find records where the First Name field is empty or does not exist.
  • Find records where the Email field contains a specific string or pattern (the "contains" condition has no value specified in the screenshot).
  • Find records where the Email field matches the regular expression pattern "[a-z]{2,}.[a-z]{2,}", which would match email addresses with at least two lowercase letters before and after the "." separator.
  • Find records where the Email field does not match the negative regular expression pattern "[0-9@#$%^&()+-:].*", which would exclude email addresses containing numbers, @ symbols, or certain special characters.
  • Find records where the Email field does not contain the string "info|marketing|admin|sales|", which appears to be filtering out common role-based email addresses.

transform-data-extract-first-and-last-name-from-email-address-step-1.png

Under Step 2, the functions are set up as follows:

  • The first function splits the Email field value by the "." delimiter and selects the 1st term from the split. This would extract the text before the first delimiter in an email address, which is typically the username or first name portion.
  • The extracted 1st term from the Email field is then formatted using the "Proper case person" format function, which capitalizes the first letter of each word (e.g., "johnsmith" becomes "John Smith").
  • The formatted value is then copied into the First Name field.
  • The second function splits the Email Username field value by the "." delimiter and selects the 2nd term from the split. This would extract the text after the first delimiter in an email address, which is often the last name portion.
  • The extracted 2nd term is formatted using the "Proper case person" format.
  • The formatted value is copied into the Last Name field.

transform-data-extract-first-and-last-name-from-email-address-step-2.png

Format on Import Using the Magical Import Module

If you have a CSV file containing data to be imported as new records, use the Magical Import module to format values in the new data as you import it.

Process Summary

  1. Upload the CSV file.
  2. Map the CSV columns to the CRM fields and specify how to import the values.
  3. Select the matching criteria.
  4. Set up the formatting rules for the imported data. 
  5. Optionally, specify other updates and modify your data before importing.
  6. Import your formatted CSV data.

 

Step-by-Step Instructions

1. Select the CSV File for Import and a Template

Navigate to Data ManagementMagical Import. Then select the database and the record type in the top menu. 

Select the CSV 

Choose the CSV file on your computer that you would like to import.

magic-import-choose-csv.png

The Preview panel will open and load the CSV rows, with a column for each field. Initially, you may see warning icons icon-red-warning-19x17.png next to columns that don't exactly match field names in your CRM database.

magical-import-contacts-preview-646w.png

2. Map CSV Columns to CRM Fields and Select Matching Criteria

AI-Powered Automatic Mapping

If you have AI enabled for your selected object type (with at least "Meta" level access configured in your AI settings), Insycle will automatically use AI to analyze your CSV column names and suggest appropriate CRM field mappings and matching criteria. This process happens in the background after your CSV loads.

The AI only accesses metadata such as column names and field labels—no actual record data is shared with AI service providers during this mapping process.

Review and Adjust Field Mappings

If AI is enabled, the Data Mapping section will automatically expand after processing your CSV. If AI is not enabled, click the Data Mapping heading to expand the section manually.

After AI processing (or when setting up manually), review how your CSV columns are mapped to CRM fields. AI will automatically map fields that it can identify with confidence. Any fields that still show a warning icon icon-red-warning-19x17.png require your attention—either AI couldn't determine the appropriate mapping, or AI is not enabled for this object type.

For unmapped fields, click the Not Mapped dropdown and search for the corresponding CRM field. You can also adjust any AI-suggested mappings if they don't match your intended use.

magical-import-contacts-map-owner.png

If columns remain unmapped, they will not be imported. You can remove a column by hovering over the column name and clicking the “X”.

Specify How to Use the Values for Each Column

Next, tell Insycle how to handle data for each field by choosing the Field Logic option for each column—update existing records, create new ones, or perform both actions based on your CRM data.

magical-import-contacts-mapping-import-mode-highlighted-646w.png

There are four Field Logic options:

  • Update: Update CRM records with all non-empty field data contained in the CSV.
  • Fill: Import CSV values only when the corresponding field in your CRM is empty.
  • Overwrite: Insycle will replace any existing data in your CRM with the data in the CSV—even if the CSV field is empty.
  • Append: Import values from the CSV and add them to the existing data in your CRM. For example, append notes from a CSV to existing notes or add to product options values on multi-select product fields (checkboxes or picklists).

Review and Confirm Fields for Matching CSV Rows to CRM Records

If AI is enabled, it will suggest Matching Criteria based on your CSV columns and CRM fields. Review these suggestions, as Matching Criteria determine how Insycle compares your CSV data with CRM records to find matches.

Matching Criteria are unique identifiers like email, phone, address, or ID number, which typically belong to one entity. For example, a different person shouldn't have Maria Smith's email or ID in their record. If a record matches any of these fields, it's recognized as Maria Smith.

You can modify AI suggestions or select the Matching Field manually. Multiple matching rules can be set and evaluated in order from top to bottom, allowing multi-criteria matching.

Note that when using an email field, Insycle automatically cross-references other email fields; similarly, it checks additional domain fields when using a domain field.

magical-import-hubspot-contacts-mapping-3-matching-rules.png

When you configure your Matching Criteria rules, the Preview will refresh, allowing you to see which records are already in your CRM. The records that Insycle found will become blue links that will open the record in your CRM. 

Validating AI Suggestions

While AI can provide an excellent starting point for column mapping and Matching Criteria, always verify that the suggestions are the best option for your specific use case and data quality standards.

Row Warnings

If multiple records in the CRM match a CSV row based on your Matching Criteria, a red warning icon will appear at the left end of the row. To learn what the problem is and determine steps to resolve it, hover over the red exclamation mark—an explanation of the error will display. Rows with errors will not be imported. 

magical-import-contacts-preview-warnings-3-matches-646w.png

3. Format the Data

Click the Data Preparation heading to expand the section.

Here you can make formatting changes and other standardization updates to the CSV data in Insycle before it is uploaded to your CRM. 

Select the Column Name from the CSV file.

Select a Function for the column, telling Insycle how the data should be formatted when imported. 

In the below example, the settings will do three things:

  1. Capitalize first and last names
  2. Standardize countries to the three-letter ISO alpha-3 codes
  3. Remove any letters from phone numbers and format them to the E.164 international standard

As you add functions, you can click Apply after each one to see the changes in the Preview

format16.png

Explore the transform options in the Function dropdown. For a complete list with descriptions, refer to the Function Catalog.

To edit the import data directly in the Preview, click the pencil icon next to any field when you hover over it.

format13.png

4. Set Up Any Other Changes or Tasks (Optional)

Optionally, you can take advantage of the many other tools and features available to manipulate your CSV data before importing it into your CRM. Learn more about using these features in the Magical Import Module Overview.

Exclude Data That Doesn’t Meet Validation Criteria

Use Data Validation to filter out records that don't meet your quality standards, or to target a segment before importing them into your CRM.

Set up validation rules and click Apply to see how it affects the CSV data in the Preview. Any rows that don't meet your criteria will display a warning icon. Hover over the icon to see specific details about why the record will be excluded.

magical-import-contacts-validation-2-email-name-country-646w.png

Make Updates to Matching CRM Records or Create Relationships

Use Data Operations to modify records in your CRM, whether by updating values on existing records or creating relationships between the CSV and CRM records. 

On the Bulk Update tab, you can add a value to CRM fields even if you don't have the field in the CSV. This will add the same value for all the imported records.

magical-import-contacts-operations-bulk-update-lifecycle-646w.png

Create relationships from the Associate/Link tab (the title will vary depending on your CRM's naming convention) between CSV records and existing records in your database during import. Learn more about creating relationships when importing.

magical-import-hubspot-contacts-operations-associate-w-companies-646w.png

Set Date Formats

When date values are detected in a CSV, Insycle determines the format being used and makes the values consistent during import.

You can override this automatic standardizing under Data Mapping using the Date Format tab. This allows you to reformat dates before importing the data into your CRM.

magical-import-contacts-mapping-date-format.png
6. Select Records and Import the Data into Your CRM

By default, Insycle will perform your chosen action on all of your CSV data. If you only want to process a few records, return to the table under Preview and check the boxes beside the selected records.

magical-import-contacts-preview-checkboxes-646w.png

At the bottom of the Magical Import page, configure the Import.

Select the Records Mode to instruct Insycle on how to handle the imported data during the import process.

  • Update existing and create net new – If Insycle is unable to find a corresponding record, a new record will be created in your CRM.
  • Only update existing – If a corresponding record is found, it will be updated with the data from your CSV import. Records that are not matched with an existing CRM record will not be imported.
  • Only create net new – Only records that can not be matched with an existing record in your database will be imported. Records that already exist in your CRM will not be updated.

When you click the Import [X] Contacts button, you'll be prompted to confirm.

⚠️ Note that there is no preview step. Once you confirm, the data will be imported into your CRM immediately.

magical-import-contacts-only-new-646w.png
7. Review the Results

After the import runs, the Import Result breaks down the import information—how many records you tried to import and how many succeeded, failed, were updated, deleted, or unmodified. Click the Run ID to open a CSV record of the import. Insycle will also send a CSV report of these changes to your email.

format19.png

Review the CSV file to see how each row of your import was handled. You can see the (Before) and (After Update) values side-by-side for each field in your import.

magical-import-salesforce-contacts-csv.png

If you see any "Failed" Results, review the Message to understand the issue and determine steps to resolve. You can also revisit any warnings shown in the Preview.

Pro Tip: Selecting Matching Criteria

Insycle uses Matching Criteria to compare your CSV to your CRM data. Matching Criteria must be "unique identifiers." These are data points that can only belong to a single entity, such as email addresses, phone numbers, street addresses, or ID numbers.

Additionally, when using an email field, Insycle will automatically cross-reference any additional email fields in the records for a match. If using a domain field, Insycle will check additional domain fields.

magical-import-contacts-mapping-import-match-criteria-email.png

 When you configure your Matching Criteria rules, the Preview will refresh, allowing you to see which records are already in your CRM. The records that Insycle found will become blue links that will open the record in your CRM.

magical-import-contacts-preview-existing-blue-links-646w.png

Important Note

You can select more than one field in each Matching Criteria rule; however, ALL of the fields must match, not just one or some of them. If you include five fields and four of them match, but one doesn't, Insycle will not consider the rule a match.

magical-import-contacts-mapping-too-many-criteria.png

In this scenario, if you import using either the 'Update existing and create net new' or 'Only create net new' Record Modes, Insycle will create a new record for any CSV rows that don't match all five criteria.

magical-import-contacts-only-new-646w.png

If you use the 'Only Update Existing' mode, there will likely be few records that match all your criteria, and much of your CSV data will not be imported because Insycle will not be able to find the correct record to update.

Typically, it is best to use a single field for your first Matching Criteria to improve the likelihood of finding existing records in your CRM. You can then add additional rules to match multiple fields.

Troubleshooting: Seeing a red warning icon in the Preview

If AI is enabled for your object type, many warning icons may automatically disappear after the initial CSV processing. Remaining warning icons indicate issues that still need your attention.

If there are issues with a CSV row, a red warning icon icon-red-warning-19x16.png will appear at the left end of the row and next to the relevant field in the Preview. Rows with errors will not be imported.

To learn what the problem is and determine steps to resolve it, hover over the red exclamation mark—an explanation of the error will display.
 

Several common reasons for the warning icon include:

  • Your validation rules
  • Several records match your criteria
  • Invalid picklist values
  • Invalid reference values
  • Unmapped columns

Your validation rules. A row will show warnings and be excluded from the operation based on your validation rules. The warning info will list which rules apply to the row.

magical-import-contacts-preview-warnings-email-country-646w.png

There are several records in your CRM that match. If multiple records have the same Matching Criteria, Insysle identifies these as matching records in your CRM.

If there should be only one record with this value, you may need to first merge duplicates and then try importing again.

If there are legitimate reasons for a Matching Field value to exist in more than one record, try adding additional Matching Criteria to make it more specific.

magical-import-contacts-preview-warnings-3-matches-646w.png

Invalid reference values. If a column is mapped to a CRM field that references other data, such as owners or associated record IDs, and no match is found, an error will appear.

magical-import-contacts-preview-warnings-invalid-owner-646w.png

Invalid picklist values. If a CRM field includes dropdown options and the data in your CSV does not match, you'll see an "Invalid picklist value" error. This value needs to be changed to match the dropdown options in your CRM.

magical-import-companies-preview-warnings-industry-picklist-646w.png

To quickly fix the import data directly in the Preview, hover over a value and click the pencil icon.

magical-import-companies-preview-warnings-industry-picklist-edit-646w.png

Once you've selected your Matching Criteria, filter options will become available in the Preview. You can use the Show Only Warning Rows filter to view only rows with warnings.

magical-import-contacts-preview-show-only-warning-rows-646w.png

Advanced How-Tos

Combining More than One Function

If there isn’t a single function that makes the change you’re looking for, you can still complete the step in one template.

Functions can be layered to make changes in a series of steps. They will apply to the field value cumulatively, executing in order from top to bottom.

For example, you can use two functions to clean up Website URL values with multiple directories. Instead of “https://app.insycle.com/data/bulk/contact/,” you just want to keep the main domain value, “insycle.com":

transform-data-contacts-clean-up-url-step-2-numbered.png

  1. The Extract: Domain from URL function will remove the https:// and subdomain, but the directory values after the domain, '/data/bulk/contact/', would remain. 
  2. To get rid of those directory values, add the function Split: By any delimiter and keep Nth term to remove everything after the forward slash, “/”, and keep the value in the first section.

For another example, you can use several functions to populate the country value by using the email address:

transform-data-hubspot-contacts-get-country-from-email-step-2-numbered.png

  1. Select the Email field with the value to start with.
  2. Select the Split: By any delimiter and pick the last term function. 
  3. In the Parameter field, enter a "." to use as the delimiter, telling Insycle which part of the email value to isolate and use for the next steps.
  4. Click the plus at the end of the row to add another function for this field.
  5. Now that you have the two-character country code, select the Standardize: Country Code2 to Name function to transform it to the full country name. Click the plus.
  6. Select the Copy: Value function and the Target Field the country name should be written into.

Troubleshooting

Phone Number Formatting In HubSpot

HubSpot has two distinct phone number features that affect how Insycle interacts with phone number data: phone number validation and dynamic phone number formatting. Understanding how these work will help you configure Insycle correctly and avoid formatting issues.

Phone Number Validation Setting

HubSpot's phone number validation can affect how Insycle can format phone numbers. When this setting is enabled, the HubSpot API only accepts phone numbers in the international standard E.164 format. For example: +48879878765 — a plus sign, country code, and digits only, with no spaces or separators. Other phone number formatting options from Insycle will result in errors.

If you're seeing errors like:

  • Underlying error message from HubSpot: Property values were not valid: [{"isValid":false,"message":"Number must match format '+18884827768' or '+18884827768 ext 123'.","error":"INVALID_PHONE_NUMBER","name":"validated_phone_number"}]
  • Underlying error message from HubSpot: Property values were not valid: [{"isValid":false,"message":"Enter a valid country code.","error":"INVALID_PHONE_NUMBER","name":"validated_phone_number"}]

You have two options to resolve this:

  1. Disable phone number validation in HubSpot — With validation turned off, HubSpot will accept phone number formats from Insycle as before.
  2. Keep validation enabled and use E.164 formatting — If you prefer to keep HubSpot's validation on, configure Insycle to format phone numbers exclusively in E.164 format.

You can find more details in HubSpot's Set up phone number property validation article.

Dynamic Phone Number Formatting

In HubSpot, fields that use the "number formatting" feature are dynamically displayed in the regional format. In Insycle, the data appears in the plain E.164 format, "+xxxxxxxxxx."

hubspot-contact-phone-number-apply-formatting-646w.png

You can format all your phone numbers using the E.164 standard in Insycle. HubSpot will then automatically add ‘number formatting’ and display the phone number in the region-specific format (the format appropriate for the region the phone number is from).

hubspot-phone-number-format-compare-w-Insycle-739w.png

Frequently Asked Questions

Which functions can I use to format names?
Function Name Description

Format: Proper case person

Format name field values with proper case.

Format: Proper case company Format company names to title case. This function can also be used to format addresses properly.

Split: By space and pick just the 1st term

Configure this to use the first name from the full name field.

Split: By space and pick just the 2nd term Configure this to use the last name from the full name field.
Which functions can I use to format phone numbers?
Function Name Description

Format: Phone E.164 Standard +xxxxxxxxxx

Format numbers to the E.164 format, which can be used for international phone number formatting.

Format: Phone USA (xxx) xxx-xxxx

Format US phone numbers to the (xxx) xxx-xxxx standard.

Format: Phone Custom Separator

Use a custom separator, such as a dash "-" or a period "." between integers and set the pattern. For example, the Phone value "442012345678" could be formatted as "44-20-1234-5678."

Which functions can I use to format addresses?
Function Name Description
Standardize: US States and CA Provinces Format the full names of US states and Canadian provinces.
Standardize: Country name to Codes or vice versa Format country names to the ISO alpha-2, two-character, or ISO alpha-3, three-character abbreviations. Or, go from the codes to the full name.
Map: US Zip Postal Code to US State Use the zip/postal code value to write the two-character US state into a specified field.
Can you help me build a custom template to format a specific field in my database?

Yes. If you're having trouble figuring out how to format a specific field, contact customer support via live chat to receive help with building your template.

transform-data-chat.png

Why do I need to filter my data?

While you can run the Transform Data module without a filter, Insycle will analyze your entire database. This is not advisable, as the operations can take a very long time for large databases. Instead, it is more efficient and manageable to filter your data into small segments and use multiple templates, rather than running operations on your entire database.

Is it best practice to build a separate template for formatting each field?

Yes. Ideally, you'll want to build separate templates for formatting each specific field. This helps you to keep things organized and troubleshoot issues if they arise.

Can I format phone numbers with country codes?

Yes, Insycle can help you format country codes. For more information, see the Format Phone Numbers for Country Codes article.

Additional Resources

Related Help Articles

Related Blog Posts