How to Format Fields in Bulk and Automatically to Ensure Data Consistency
Data within an organization often originates from multiple sources, including website forms, internal data entry, integrations, and APIs. This can make it challenging to enforce consistent formatting across specific fields, complicating efforts to segment and search for records effectively.
With Insycle, you can format any field in your database using pre-defined rulesets and automate those templates to ensure your fields are consistently formatted correctly.
Existing data in any field can be formatted in bulk using the Transform Data module. Alternatively, if you have an export from a third-party application or other data in a CSV format, the new data can be formatted on import using the Magical Import module.
Format in Bulk Using the Transform Data Module
The Transform Data module is ideal for formatting fields like names, phone numbers, and addresses. You can easily format and standardize any free-text field in your CRM in bulk.
Process Summary
- Find and review records for formatting.
- Set up functions to format field values.
- Preview the changes.
- Apply changes to the CRM records.
Step-by-Step Instructions
Navigate to Data Management > Transform Data, pick a record type, and explore the default templates for a pre-built solution.
Under Step 1, adjust the settings to filter the CRM records down to those you would like to format. This ensures you aren't trying to format fields that don't contain any data. And it is best practice to only tackle one field at a time.
The example below will search for contacts with the first name in all caps:
- Field: First Name
- Condition: regex
- Value: [A-Z]{3,}
This regular expression is looking for:
- Uppercase letters from A to Z: [A-Z]
- Three or more occurrences of these uppercase letters: {3,}
You can add any filter relevant to your use case—lifecycle stages, record creation dates, engagement triggers, etc. Insycle can use any data in your CRM in the filter step.
Click the Search button and scroll down to the Record Viewer at the bottom of the screen. All records that match your filter will appear. You can alter the fields that appear in this preview from the Layout tab in Step 1.
Under Step 2, give Insycle instructions on what formatting and cleanup changes to make to the identified records.
Select the Field that contains the value you want to start with, then select the Function and enter the parameters. Click the plus at the end of the row to add an additional function for this field.
Each function will use the value output from the previous function, so their sequence matters.
There are pre-built functions for standardizing phone numbers, countries, states, domains, and other values, so you should explore the options in the Function dropdown or review the Function Catalog.
In the above example, these standardizations are being made to the First Name in each contact record:
- Remove whitespace from before or after the value.
- Remove salutations such as Mr or Mr. or Mrs or Mrs.
- Capitalize the first name value.
The next example will format specific parts of the Street Address field values. The "Map: Terms" Function will look for the Existing Text and if found, will change it to the New Text value. Here in the Existing Text fields, the bar character "|" between the values means OR, as in, look for "street" or "st.".
Altogether, these rules will standardize the Street Address like this:
- Street → St
- Drive → Dr
- Avenue → Ave
- Boulevard → Blvd
Preview Changes in the CSV Report
With the filters and functions set up, you can preview the changes. It's important to verify that your formatting is working as expected before those changes are pushed to your live database.
Under Step 3, click the Review button, then select the Preview option.
On the Notify tab, add any additional recipients who should receive the CSV (and make sure to hit Enter after each address). You can add additional context to the subject line and email body.
On the When tab, click Run Now and select which records to apply the change to (you could do All, but if you have a large number of records, you may just want to do a chunk for your preview), then click the Run Now button.
Open the CSV file from your email in a spreadsheet application and review the values for each row.
The Result column indicates the outcome of the operation for each record. The possible results are:
- Updated - The transform was applied successfully.
- Failed - The transform could not be applied. See the Message column for details.
- Unmodified - The transform was not applicable to the record.
To the far right, you can see the (Before) and (After) values side-by-side; this example shows First Name (Before) and First Name (After).
If the results don't look as you expected, go back to your filters in Step 1 and functions in Step 2, and try making some adjustments. Then, preview again.
Apply Changes to the CRM
If everything in your CSV preview looks correct, return to Insycle and proceed with applying the changes to the live CRM data.
Under Step 3, click the Review button again, and this time select Update mode.
On the When tab, you should use Run Now the first time you apply these changes to the CRM. If you have a large number of records, you may want to do a smaller batch to review the results in your CRM.
After you've seen the results in the CRM and you are satisfied with how the operation runs, you can save all of the configurations as a template, and set up automation so this formatting operation runs on a set schedule.
If you have several templates you'd like to run together automatically, you can create a Recipe. Additionally, HubSpot users can integrate Insycle Recipes into HubSpot Workflows.
By automating with a template, you'll ensure that your fields are consistently and automatically formatted on an ongoing basis.
You can combine functions to isolate parts of a value, and apply formatting before copying to a destination field.
Extract First and Last Name from email address
You can look for email addresses that follow the "firstname.lastname@domain.com" format, then combine functions to parse and extract potential first and last name values from email addresses, format them properly with capitalization, and populate the respective First Name and Last Name fields accordingly.
In this example, the filter under Step 1 is set up to:
- Find records where the First Name field is empty or does not exist.
- Find records where the Email field contains a specific string or pattern (the "contains" condition has no value specified in the screenshot).
- Find records where the Email field matches the regular expression pattern "[a-z]{2,}.[a-z]{2,}", which would match email addresses with at least two lowercase letters before and after the "." separator.
- Find records where the Email field does not match the negative regular expression pattern "[0-9@#$%^&()+-:].*", which would exclude email addresses containing numbers, @ symbols, or certain special characters.
- Find records where the Email field does not contain the string "info|marketing|admin|sales|", which appears to be filtering out common role-based email addresses.
Under Step 2, the functions are set up as follows:
- The first function splits the Email field value by the "." delimiter and selects the 1st term from the split. This would extract the text before the first delimiter in an email address, which is typically the username or first name portion.
- The extracted 1st term from the Email field is then formatted using the "Proper case person" format function, which capitalizes the first letter of each word (e.g., "johnsmith" becomes "John Smith").
- The formatted value is then copied into the First Name field.
- The second function splits the Email Username field value by the "." delimiter and selects the 2nd term from the split. This would extract the text after the first delimiter in an email address, which is often the last name portion.
- The extracted 2nd term is formatted using the "Proper case person" format.
- The formatted value is copied into the Last Name field.
Format on Import Using the Magical Import Module
If you have a CSV file containing data to be imported as new records, use the Magical Import module to format values in the new data as you import it.
Process Summary
- Upload the CSV file.
- Map the CSV columns to the CRM fields and specify how to import the values.
- Select the matching criteria.
- Set up the formatting rules for the imported data.
- Optionally, specify other updates and modify your data before importing.
- Import your formatted CSV data.
Step-by-Step Instructions
Navigate to Data Management > Magical Import. Then select the database and the record type in the top menu.
Select the CSV
Choose the CSV file on your computer that you would like to import.
The Preview panel will open and load the CSV rows, with a column for each field. Initially, you may see warning icons next to columns that don't exactly match field names in your CRM database.
AI-Powered Automatic Mapping
If you have AI enabled for your selected object type (with at least "Meta" level access configured in your AI settings), Insycle will automatically use AI to analyze your CSV column names and suggest appropriate CRM field mappings and matching criteria. This process happens in the background after your CSV loads.
The AI only accesses metadata such as column names and field labels—no actual record data is shared with AI service providers during this mapping process.
Review and Adjust Field Mappings
If AI is enabled, the Data Mapping section will automatically expand after processing your CSV. If AI is not enabled, click the Data Mapping heading to expand the section manually.
After AI processing (or when setting up manually), review how your CSV columns are mapped to CRM fields. AI will automatically map fields that it can identify with confidence. Any fields that still show a warning icon require your attention—either AI couldn't determine the appropriate mapping, or AI is not enabled for this object type.
For unmapped fields, click the Not Mapped dropdown and search for the corresponding CRM field. You can also adjust any AI-suggested mappings if they don't match your intended use.
If columns remain unmapped, they will not be imported. You can remove a column by hovering over the column name and clicking the “X”.
Specify How to Use the Values for Each Column
Next, tell Insycle how to handle data for each field by choosing the Field Logic option for each column—update existing records, create new ones, or perform both actions based on your CRM data.
There are four Field Logic options:
- Update: Update CRM records with all non-empty field data contained in the CSV.
- Fill: Import CSV values only when the corresponding field in your CRM is empty.
- Overwrite: Insycle will replace any existing data in your CRM with the data in the CSV—even if the CSV field is empty.
- Append: Import values from the CSV and add them to the existing data in your CRM. For example, append notes from a CSV to existing notes or add to product options values on multi-select product fields (checkboxes or picklists).
Review and Confirm Fields for Matching CSV Rows to CRM Records
If AI is enabled, it will suggest Matching Criteria based on your CSV columns and CRM fields. Review these suggestions, as Matching Criteria determine how Insycle compares your CSV data with CRM records to find matches.
Matching Criteria are unique identifiers like email, phone, address, or ID number, which typically belong to one entity. For example, a different person shouldn't have Maria Smith's email or ID in their record. If a record matches any of these fields, it's recognized as Maria Smith.
You can modify AI suggestions or select the Matching Field manually. Multiple matching rules can be set and evaluated in order from top to bottom, allowing multi-criteria matching.
Note that when using an email field, Insycle automatically cross-references other email fields; similarly, it checks additional domain fields when using a domain field.
When you configure your Matching Criteria rules, the Preview will refresh, allowing you to see which records are already in your CRM. The records that Insycle found will become blue links that will open the record in your CRM.
Validating AI Suggestions
While AI can provide an excellent starting point for column mapping and Matching Criteria, always verify that the suggestions are the best option for your specific use case and data quality standards.
Row Warnings
If multiple records in the CRM match a CSV row based on your Matching Criteria, a red warning icon will appear at the left end of the row. To learn what the problem is and determine steps to resolve it, hover over the red exclamation mark—an explanation of the error will display. Rows with errors will not be imported.
Click the Data Preparation heading to expand the section.
Here you can make formatting changes and other standardization updates to the CSV data in Insycle before it is uploaded to your CRM.
Select the Column Name from the CSV file.
Select a Function for the column, telling Insycle how the data should be formatted when imported.
In the below example, the settings will do three things:
- Capitalize first and last names
- Standardize countries to the three-letter ISO alpha-3 codes
- Remove any letters from phone numbers and format them to the E.164 international standard
As you add functions, you can click Apply after each one to see the changes in the Preview.
Explore the transform options in the Function dropdown. For a complete list with descriptions, refer to the Function Catalog.
To edit the import data directly in the Preview, click the pencil icon next to any field when you hover over it.
Optionally, you can take advantage of the many other tools and features available to manipulate your CSV data before importing it into your CRM. Learn more about using these features in the Magical Import Module Overview.
Exclude Data That Doesn’t Meet Validation Criteria
Use Data Validation to filter out records that don't meet your quality standards, or to target a segment before importing them into your CRM.
Set up validation rules and click Apply to see how it affects the CSV data in the Preview. Any rows that don't meet your criteria will display a warning icon. Hover over the icon to see specific details about why the record will be excluded.
Make Updates to Matching CRM Records or Create Relationships
Use Data Operations to modify records in your CRM, whether by updating values on existing records or creating relationships between the CSV and CRM records.
On the Bulk Update tab, you can add a value to CRM fields even if you don't have the field in the CSV. This will add the same value for all the imported records.
Create relationships from the Associate/Link tab (the title will vary depending on your CRM's naming convention) between CSV records and existing records in your database during import. Learn more about creating relationships when importing.
Set Date Formats
When date values are detected in a CSV, Insycle determines the format being used and makes the values consistent during import.
You can override this automatic standardizing under Data Mapping using the Date Format tab. This allows you to reformat dates before importing the data into your CRM.
By default, Insycle will perform your chosen action on all of your CSV data. If you only want to process a few records, return to the table under Preview and check the boxes beside the selected records.
At the bottom of the Magical Import page, configure the Import.
Select the Records Mode to instruct Insycle on how to handle the imported data during the import process.
- Update existing and create net new – If Insycle is unable to find a corresponding record, a new record will be created in your CRM.
- Only update existing – If a corresponding record is found, it will be updated with the data from your CSV import. Records that are not matched with an existing CRM record will not be imported.
- Only create net new – Only records that can not be matched with an existing record in your database will be imported. Records that already exist in your CRM will not be updated.
When you click the Import [X] Contacts button, you'll be prompted to confirm.
⚠️ Note that there is no preview step. Once you confirm, the data will be imported into your CRM immediately.
After the import runs, the Import Result breaks down the import information—how many records you tried to import and how many succeeded, failed, were updated, deleted, or unmodified. Click the Run ID to open a CSV record of the import. Insycle will also send a CSV report of these changes to your email.
Review the CSV file to see how each row of your import was handled. You can see the (Before) and (After Update) values side-by-side for each field in your import.
If you see any "Failed" Results, review the Message to understand the issue and determine steps to resolve. You can also revisit any warnings shown in the Preview.
Insycle uses Matching Criteria to compare your CSV to your CRM data. Matching Criteria must be "unique identifiers." These are data points that can only belong to a single entity, such as email addresses, phone numbers, street addresses, or ID numbers.
Additionally, when using an email field, Insycle will automatically cross-reference any additional email fields in the records for a match. If using a domain field, Insycle will check additional domain fields.
When you configure your Matching Criteria rules, the Preview will refresh, allowing you to see which records are already in your CRM. The records that Insycle found will become blue links that will open the record in your CRM.
Important Note
You can select more than one field in each Matching Criteria rule; however, ALL of the fields must match, not just one or some of them. If you include five fields and four of them match, but one doesn't, Insycle will not consider the rule a match.
In this scenario, if you import using either the 'Update existing and create net new' or 'Only create net new' Record Modes, Insycle will create a new record for any CSV rows that don't match all five criteria.
If you use the 'Only Update Existing' mode, there will likely be few records that match all your criteria, and much of your CSV data will not be imported because Insycle will not be able to find the correct record to update.
Typically, it is best to use a single field for your first Matching Criteria to improve the likelihood of finding existing records in your CRM. You can then add additional rules to match multiple fields.
If AI is enabled for your object type, many warning icons may automatically disappear after the initial CSV processing. Remaining warning icons indicate issues that still need your attention.
If there are issues with a CSV row, a red warning icon will appear at the left end of the row and next to the relevant field in the Preview. Rows with errors will not be imported.
To learn what the problem is and determine steps to resolve it, hover over the red exclamation mark—an explanation of the error will display.
Several common reasons for the warning icon include:
- Your validation rules
- Several records match your criteria
- Invalid picklist values
- Invalid reference values
- Unmapped columns
Your validation rules. A row will show warnings and be excluded from the operation based on your validation rules. The warning info will list which rules apply to the row.
There are several records in your CRM that match. If multiple records have the same Matching Criteria, Insysle identifies these as matching records in your CRM.
If there should be only one record with this value, you may need to first merge duplicates and then try importing again.
If there are legitimate reasons for a Matching Field value to exist in more than one record, try adding additional Matching Criteria to make it more specific.
Invalid reference values. If a column is mapped to a CRM field that references other data, such as owners or associated record IDs, and no match is found, an error will appear.
Invalid picklist values. If a CRM field includes dropdown options and the data in your CSV does not match, you'll see an "Invalid picklist value" error. This value needs to be changed to match the dropdown options in your CRM.
To quickly fix the import data directly in the Preview, hover over a value and click the pencil icon.
Once you've selected your Matching Criteria, filter options will become available in the Preview. You can use the Show Only Warning Rows filter to view only rows with warnings.
Advanced How-Tos
If there isn’t a single function that makes the change you’re looking for, you can still complete the step in one template.
Functions can be layered to make changes in a series of steps. They will apply to the field value cumulatively, executing in order from top to bottom.
For example, you can use two functions to clean up Website URL values with multiple directories. Instead of “https://app.insycle.com/data/bulk/contact/,” you just want to keep the main domain value, “insycle.com":
- The Extract: Domain from URL function will remove the https:// and subdomain, but the directory values after the domain, '/data/bulk/contact/', would remain.
- To get rid of those directory values, add the function Split: By any delimiter and keep Nth term to remove everything after the forward slash, “/”, and keep the value in the first section.
For another example, you can use several functions to populate the country value by using the email address:
- Select the Email field with the value to start with.
- Select the Split: By any delimiter and pick the last term function.
- In the Parameter field, enter a "." to use as the delimiter, telling Insycle which part of the email value to isolate and use for the next steps.
- Click the plus at the end of the row to add another function for this field.
- Now that you have the two-character country code, select the Standardize: Country Code2 to Name function to transform it to the full country name. Click the plus.
- Select the Copy: Value function and the Target Field the country name should be written into.
Troubleshooting
HubSpot has two distinct phone number features that affect how Insycle interacts with phone number data: phone number validation and dynamic phone number formatting. Understanding how these work will help you configure Insycle correctly and avoid formatting issues.
Phone Number Validation Setting
HubSpot's phone number validation can affect how Insycle can format phone numbers. When this setting is enabled, the HubSpot API only accepts phone numbers in the international standard E.164 format. For example: +48879878765 — a plus sign, country code, and digits only, with no spaces or separators. Other phone number formatting options from Insycle will result in errors.
If you're seeing errors like:
- Underlying error message from HubSpot: Property values were not valid: [{"isValid":false,"message":"Number must match format '+18884827768' or '+18884827768 ext 123'.","error":"INVALID_PHONE_NUMBER","name":"validated_phone_number"}]
- Underlying error message from HubSpot: Property values were not valid: [{"isValid":false,"message":"Enter a valid country code.","error":"INVALID_PHONE_NUMBER","name":"validated_phone_number"}]
You have two options to resolve this:
- Disable phone number validation in HubSpot — With validation turned off, HubSpot will accept phone number formats from Insycle as before.
- Keep validation enabled and use E.164 formatting — If you prefer to keep HubSpot's validation on, configure Insycle to format phone numbers exclusively in E.164 format.
You can find more details in HubSpot's Set up phone number property validation article.
Dynamic Phone Number Formatting
In HubSpot, fields that use the "number formatting" feature are dynamically displayed in the regional format. In Insycle, the data appears in the plain E.164 format, "+xxxxxxxxxx."
You can format all your phone numbers using the E.164 standard in Insycle. HubSpot will then automatically add ‘number formatting’ and display the phone number in the region-specific format (the format appropriate for the region the phone number is from).
Frequently Asked Questions
| Function Name | Description |
|---|---|
Format: Proper case person |
Format name field values with proper case. |
| Format: Proper case company | Format company names to title case. This function can also be used to format addresses properly. |
Split: By space and pick just the 1st term |
Configure this to use the first name from the full name field. |
| Split: By space and pick just the 2nd term | Configure this to use the last name from the full name field. |
| Function Name | Description |
|---|---|
Format: Phone E.164 Standard +xxxxxxxxxx |
Format numbers to the E.164 format, which can be used for international phone number formatting. |
Format: Phone USA (xxx) xxx-xxxx |
Format US phone numbers to the (xxx) xxx-xxxx standard. |
Format: Phone Custom Separator |
Use a custom separator, such as a dash "-" or a period "." between integers and set the pattern. For example, the Phone value "442012345678" could be formatted as "44-20-1234-5678." |
| Function Name | Description |
|---|---|
| Standardize: US States and CA Provinces | Format the full names of US states and Canadian provinces. |
| Standardize: Country name to Codes or vice versa | Format country names to the ISO alpha-2, two-character, or ISO alpha-3, three-character abbreviations. Or, go from the codes to the full name. |
| Map: US Zip Postal Code to US State | Use the zip/postal code value to write the two-character US state into a specified field. |
Yes. If you're having trouble figuring out how to format a specific field, contact customer support via live chat to receive help with building your template.
While you can run the Transform Data module without a filter, Insycle will analyze your entire database. This is not advisable, as the operations can take a very long time for large databases. Instead, it is more efficient and manageable to filter your data into small segments and use multiple templates, rather than running operations on your entire database.
Yes. Ideally, you'll want to build separate templates for formatting each specific field. This helps you to keep things organized and troubleshoot issues if they arise.
Yes, Insycle can help you format country codes. For more information, see the Format Phone Numbers for Country Codes article.
Additional Resources
Related Help Articles
- Format Phone Numbers for Country Codes
- Standardize Deal Name Format Using Values From Multiple Fields
- Map Values From One Field to Another
- Module Overview: Transform Data
- Standardize Job Title, Industry, State, Country, or Any Other Free-Text Field
Related Blog Posts