Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Mappings are at the heart of Patchworks.
When we pull data from one system and push it into another, it’s unlikely that the two will have a like-for-like data structure. By creating mappings between source and target data fields, the Patchworks engine knows how to transform incoming data as needed to update the target system.
The illustration below helps to visualise this:
In process flows, the map shape is used to define how data fields pulled from one connector correlate with data fields in another connector, and whether any data transformations are required to achieve this.
If your organisation has in-house development expertise and complex transformation requirements, you can use our custom scripts feature to code your own scripts for use with field mappings.
The map shape includes everything that you need to map data fields between two connections in a process flow. When you start to create mappings, there are two approaches to consider:
Having added a map shape to a process flow, click the associated 'cog' icon to access settings:
For more information about working with these settings, please see our Working with field mappings page.
The generate automatic mapping option is used to auto-generate mappings between your selected source and target connections.
All Patchworks prebuilt connectors (i.e. connectors installed from the Patchworks marketplace) adopt a standardised taxonomy for tagging common fields found in data schemas for a range of entity types (customers, orders, refunds, products, fulfillments, etc.). So, if your process flow includes connections to sync data between two prebuilt connectors, it's highly likely that auto-generating mappings will complete a lot of the work for you.
Once auto-generation is complete, mapping rows are added for all fields found in the source data - for example:
Where matching tags are found, the mapping rows will include both source and target fields (you can adjust these manually and/or apply transformations, as needed).
Any fields found in the source data which could not be matched by tag are displayed in partial mapping rows, ready for you to add a target manually.
For more information about using the generate automatic mapping feature, please see our Working with field mappings page.
If your process flow includes custom connections (i.e. connectors that have been built by your organisation, using the Patchworks connector builder), you can still use the generate automatic mapping option. The success of this will depend on whether field tagging was applied to your connector during the build:
If yes, your custom connector will behave like any of our prebuilt connectors when it comes to auto-generated mappings, adding fully mapped rows for all matched tags.
If no, Patchworks won't be able to match any source fields to a target automatically - partial mapping rows are added for all source fields found, ready for you to add a target manually.
It's very easy to add individual mapping rows manually, using the add mapping rule option:
We recommend that you always try the automatic mapping option first and then manually add any extra rows if needed. However, there's no reason that you couldn't add all of your mappings manually if preferred.
For more information about adding mappings manually, please see our Working with field mappings page.
If you're building multiple process flows with similar requirements for field mappings, you can export the configuration for a map shape, and then import that configuration into another map shape.
When a map shape configuration is exported, a JSON file is generated and saved (automatically) to the default download folder for your browser. All field mappings and associated transformations are exported. You can then import this file to any other map shape within:
The same process flow
Other process flows for your company profile
Other process flows for any of your linked company profiles
To export the configuration for a map shape, follow the steps below:
Step 1 Access the required process flow, then click the settings icon for the map shape that you want to export:
Step 2 Click the export map button:
Step 3
The configuration is exported and saved to your default downloads folder. The filename is always map.json
.
To import a mapping configuration into a map shape, follow the steps below:
Step 1 Access the required process flow, then click the settings icon for the map shape that you want to update:
You can import a mapping configuration into a new map shape, or into an existing one. If you import a configuration into an existing map shape, any existing mappings will be overwritten.
Step 2 Click the import map button:
Step 3 Navigate to the downloaded map configuration file on your local drive, then select it.
The default filename for exported map configuration files is map.json
.
Step 4 Having selected a valid mapping configuration file to import, the import completes immediately.
This page provides guidance on using the map shape to configure field mappings between two connections.
Step 1 Click the source endpoint option:
...source and target selection fields are displayed:
Step 2 Use source and target selection fields to choose the required connector instance and associated endpoints to be mapped - for example:
Step 3 Click the generate automatic mapping button:
...when prompted to confirm this operation, click generate mapping:
As we're configuring a new map shape, there's no danger that we would overwrite existing mappings. However, always use this option with caution if you're working with an existing map shape - any existing mapping rules are overwritten when you choose to generate automatic mappings.
If you need to access the generate automatic option for an existing mapping shape, you need to click into the source and target details first.
Step 4 Patchworks attempts to apply mappings between your given source and target automatically. A mapping rule is added for each source data field and, where possible, a matched target field - for example:
From here you can refine mappings as needed. You can:
Step 5
Toggle wrap input payload
and wrap output payload
options ON/OFF as required.
Where:
wrap input payload
ON. Wraps the incoming payload in an array [ ]
ONLY for processing within the map shape.
wrap output payload
ON. Wraps the outgoing payload in an array [ ]
ONLY for onward processing.
Click options below for payload examples showing how these options work in practice:
Step 6 Save changes.
You can add as many new mapping rules as required to map data between source and target connections.
There may be times where you don't want to (or can't) use the payload fields dropdown select a field from your source/target data schema. In this case, you simply select the manual input field and enter the full schema path for the required field.
You can change the display name and/or the field associated with the source or target for any mapping rule.
If you've used the automatically generate map option to generate an initial set of mappings, you may find that some source fields could not be auto-mapped. In these cases, a mapping rule is added for each un-mapped source field, so you can either add the required destination or delete the rule.
If required, you can map a source field to multiple target fields - for example, you might need to send a customer order number into two (or more) target fields.
Sometimes it can be useful to map multiple source fields to a single target field. For example, you might have a target connection which expects a single field for 'full name', but a source connection with one field for 'first name' and another field for 'surname'.
In this case, you would define mappings for the required source and target fields, then add a transform function to concatenate the two source fields.
When you choose to delete a mapping rule, it's removed from the list immediately. However, the deletion is not permanent until you choose to save the mapping shape.
The custom dynamic date transform function is used to set a target field to the current date and time, based on the date and time that the process flow runs. You can also define rounding and adjustments.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select custom dynamic date:
Step 5 Optionally, you can add adjustment settings - for example:
These options are summarised below:
Step 6 Accept your changes:
...then save the transformation:
Step 7 Now you can select a target field in the usual way - for example:
...then:
...then:
Step 8 Once your mapping is complete, the row should be displayed without a source field - for example:
From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the custom dynamic date will be mapped to the given target field.
Field transformations can be defined to change the value of a data field pulled from a source system before it is sent to its target. A transformation is comprised of one or more .
This page explains how to for a field mapping, and .
For a summary of available transform functions please see the section.
For information about adding a field transformation using a cross-reference lookup table, please see our section.
To add a new transformation for a field mapping, you start by adding a new transformation and then build the required functions. To do this, follow the steps below:
Step 1 Access the required process flow and then edit the map shape to be updated with a transform:
Step 2 Find the mapping that you want to update, then click the transform icon (between source and target elements). For our example, we're going to add a prefix to the 'id' field:
Step 3 Click the add transform button:
Step 4 Use the select a function field to choose the type of function that you need to use (functions are organised by type):
Step 5 Depending on the type of function you select, additional fields are displayed for you to complete. Update these as required - for our example we're adding some text to be added as a prefix:
Step 6 Now we need to confirm which source field this transform field should be applied for - click the add field button:
Step 7 Select the required field:
In straightforward scenarios, this will typically be the same source field as defined for the mapping row. However, more complex scenarios may prompt multiple options here - for example, if you apply multiple transforms to the same mapping.
Step 8 Accept your changes.
Step 9 Add more fields if necessary.
Step 10 When you're satisfied that all required fields have been added, accept changes and then save the shape settings.
The array join transform function is used to join elements of an array as a string, with a user-defined delimiter. For example, you might have product data in an array:
...and need to convert these items to a string before pushing to a single destination field (with each item in the string delimited by a character of your choice):
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select join from the array category:
Step 5 In the delimiter field, enter the character that you want to use to delimit each array field in the string:
A space is a valid character.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The custom static date transform function is used to set a target field to a given date and time.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select custom static date:
Step 5 Click anywhere in the date field, or click the calendar icon, to open a date picker:
Step 6 Set the required date and time.
Step 7 Accept your changes:
...then save the transformation:
Step 8 Now you can select a target field in the usual way - for example:
...then:
...then:
Step 9 Once your mapping is complete, the row should be displayed without a source field - for example:
From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the custom static date will be mapped to the given target field.
Any instances defined for your company profile are available to select as the source or target. If you aren't using a connector to retrieve data (for example, you are sending in data via the Inbound API or a webhook), you won't select a source endpoint - instead, use the override source format dropdown field to select the format of your incoming data:
Field | Summary | Example |
---|
Transform | Description |
---|
Transform | Description |
---|
Transform | Description / Notes |
---|
Round |
| Suppose that the process flow runs at the following date/time: 2023-08-10 10:30:0
If set to |
Units | If you want to adjust the date/time, select the required unit - choose from | Suppose that the process flow runs at the following date/time: 2023-08-10 10:30:0 and you want to adjust it by 1 day.
In this case, you would select |
Adjustment | Having selected an adjustment unit, enter the required number of that unit here. | See |
Join elements of an array as a string, based on a defined delimiter. |
Apply the date that a process flow runs, with or without adjustments. |
Apply a static date. |
Convert a date to a predefined or custom format. |
Round a date up/down to the start/end of the day. |
Time now | Returns the current date and time in your required format. |
Timezone | Convert dates to a selected timezone. |
Cast to boolean | Convert a number value to a boolean (true/false) value based on PHP logic. |
Change the source field data type from |
Ceiling | Round up to the nearest whole number. |
Apply a static number. |
Floor | Round down to the nearest whole number. |
Make negative | Convert number to a negative. |
Make positive | Convert number to a positive. |
Perform a mathematical operation for selected fields. |
Change the number of decimal places. |
Define override values for conversion to true/false. |
Change the source field data type from |
Reference a value from cached data. |
Convert weight | Convert a specified weight unit to a given alternative. |
Apply a true or false value. |
Fallback | Set a default value to be used if the given input is empty. Blank values are supported. |
Map |
Convert a null value to an empty string. |
Convert a null value to zero (0). |
Convert a source value to null. |
Change the source field data type from |
Change the source field data type from |
Join selected fields with a selected character. |
Specify a comma separated list of field values to be matched for inclusion. |
Convert string to boolean | Convert a string value to a boolean (true/false) value based on PHP logic. |
Country code | Apply country codes of a selected type (Alpha 1, Alpha 2, Numeric). Note that this transform will cause the map step to fail if an empty value is received. |
Country name | Return the country name for a country code. Note that this transform will cause the map step to fail if an empty value is received. |
Apply static text or reference variables. |
Specify a comma separated list of field values to be matched for exclusion. |
Get the first word from a string. |
Hash | Convert a string to a SHA1 Hash. |
Encode data into JSON format. Note that although this is listed as a |
Get the last word from a string. |
Limit | Truncates a string to a given length. |
Lowercase | Convert to lowercase. |
Pad an existing string of characters to a given length, using a given character. |
Prefix | Add a string to the beginning of a field. |
Replace any given character(s) with another character. |
Split elements of a string into an array. |
Substring | Return a given number of characters from a given start. |
Suffix | Add a string to the end of a field. |
Trim whitespace | Remove any characters around a string. |
Uppercase | Convert to uppercase. |
URL decode | Convert an encoded URL into a readable format. |
URL encode | Convert a string to a URL encoded string. |
The round number transform function is used to change the number of decimal places for a number value. For example:
...might be changed to:
With the round number transform you can specify the number of decimal places that should be applied to incoming numeric values.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select round number from the number category:
Step 5 Move to the decimal places field and enter the required number of decimal places required for transformed values - for example:
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The custom number transform function is used to map a given number to a target field.
If you've added/updated a map shape before, you'll be used to selecting a source field and a target field. However, when a custom number transformation is used we don't select a source field - the custom number transformation is our data source.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select custom number:
Step 5 Move down to the custom number field and enter your required number - for example:
Step 6 Accept your changes:
...then save the transformation:
Step 7 Now you can select a target field in the usual way - for example:
...then:
...then:
Step 8 Once your mapping is complete, the row should be displayed without a source field - for example:
From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the custom number will be mapped to the given target field.
The format transform function is used to change a date value to a different format. For example:
...might be changed to:
A range of predefined date formats is available for selection, or you can set your own custom format.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select format from the date category:
Step 5 Click in the format field to select a predefined date format that incoming dates should be converted to:
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
Internally, the format transform function uses Laravel's date format methods, which in turn call PHP date format methods. Commonly used format specifiers are listed below - full details are available in this Laravel guide.
The following characters are commonly used to specify days in custom format dates.
The following characters are commonly used to specify months in custom format dates.
The following characters are commonly used to specify years in custom format dates.
The following characters are commonly used to specify times in custom format dates.
Unix Epoch dates must be received as a number, not a string - i.e.:
1701734400
rather than "1701734400"
If your Unix dates are provided as strings, you should convert these to numbers. To achieve this, add a cast to number transform for the date field BEFORE the date format transform function.
The cast to string transform function is used to change the data type associated with a source field from number
to string
. For example, you might have an id
field in a source system that's stored as string
value, but your destination system expects the id
to be a number
.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select cast to string from the number category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The math transform function is used to perform a mathematical operation for selected fields. For example, your incoming payload might include customer records, each with a series of numeric value
fields that need to be added together so the total can be pushed to a total
field in the target system.
The following mathematical operations are available:
Add
Subtract
Multiply
Divide
In the instructions below, we'll step through the scenario mentioned above where our incoming payload includes customer records, each with three value fields (value1
, value2
, value3
) that must be added together and pushed to a total
field in the target system.
The steps required are detailed in two stages:
To begin, we need to update/add the required mapping row so that it includes all source fields that need to be added together and then pushed to the target.
Step 1 In your process flow, access settings for your map shape:
Step 2
Find (or add) the mapping row which requires a math transformation. In the example below, we have a row that's currently set to map the source first name
field into the destination full name
field:
Step 3 On the source side of the mapping row, we need to include all the fields to be used in our mathematical operation. To do this, click the 'pencil' icon associated with the existing source field:
Step 4 Details for the selected field are shown - click the add source field button:
Step 5 Click the 'pencil' icon associated with the new source field:
Step 6 Move down and update the display name and payload fields for the second source field that you want to use - for example:
Step 7 Accept these changes to exit back to your mapping rows - notice that there are now two source fields associated with the row you updated:
Step 8 Repeat steps 3 to 7 to add any more source fields that you need to include in the mathematical operation.
With all required source fields defined for our mapping row, we can add a math transform function to define the required calculation based on these fields.
Step 1 Select the add transform button for the required mapping rule - for example:
Step 2 Click the add transform button:
Step 3 Click in the name field and select math from the number section in the list of transform functions:
...math options are displayed:
Step 4 Click in the operator field and select the type of calculation to be performed - you can choose from add, subtract, multiply and divide:
Step 5 Click the add field button:
Step 6 Click in source fields and select the first source field to be used in the calculation:
Step 7 Accept your changes.
Step 8 Click the add field button again:
...and add the next source field to be used - for example:
Step 9 Accept your changes.
Step 10 Repeat steps 8 and 9 to include any more source fields to be used in the calculation. Each time you accept a new source field you'll see the sequence that they will be processed when this transform function runs - for example:
Fields are processed in the sequence that they are added here.
Step 11 Having added all required source fields to be calculated, accept changes:
...then save the function:
Step 12 Ensure that the target field for this mapping row is set as required, then save the map shape. Next time the process flow runs, the mathematical operation will be performed for the given source fields and the total value is pushed to the defined target field. The example below shows an incoming payload before and after the math transformation is applied:
The round date transform function is used to round source dates to either the start or end of the day, where:
start of day
changes the time to 00:00
for the received date
end of day
changes the time to 23:59
for the received date
So, you can round a given source date before sending the rounded value into a given target field.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select round date:
Step 5 Choose your required rounding:
Step 6 Accept your changes and save the transformation - at this point your mapping row is displayed without a target. From here, you can go ahead and add a target field:
The cast boolean to string transform function is used to change the data type associated with a source field from boolean
to string
.
A boolean
data type can have only two possible states: true
or false
.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select cast boolean to string from the other category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The cast to boolean transform function is used to convert given values to true
/false
based on PHP logic, but with the option to define overrides for specific input values. For example, consider the following payload:
Suppose we want to define a mapping for the fruit_included
item (line 2) but convert the numeric value to a true
/false
(boolean) value. Without an override setting, transforming the number to a boolean value would result in the following payload:
This is because standard PHP logic determines that 0
equates to a boolean false
value. But in this specific case, a quirk in our source data is such that the 0
value actually means true
- hence a list of fruits follows.
So, we need a way to specify an override for this field, where 0
equates to true
. We can do this via the other > cast to boolean transform function, thereby achieving the desired result:
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select cast to boolean from the other category:
Step 5
In the true values (override)
and/or false values (override)
fields, enter specific values that should override standard PHP logic:
Multiple override values can be entered - use a comma to separate each one.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The custom boolean transform function is used to map a value of true
or false
to a target field.
If you've added/updated a before, you'll be used to selecting a source field and a target field. However, when a custom boolean transformation is used we don't select a source field - the custom boolean transformation is our data source.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select custom boolean:
Step 5 Move down to the value field and select your required true/false value - for example:
Step 6 Accept your changes:
...then save the transformation:
Step 7 Now you can select a target field in the usual way - for example:
...then:
...then:
Step 8 Once your mapping is complete, the row should be displayed without a source field - for example:
From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the selected custom boolean value will be mapped to the given target field.
The script transform function is used to apply an existing to the source value, and the updated field value is pushed to the target field.
Make sure that the required script has been before applying it as a transform function.
Any payloads passed in and out of a script transform are verbatim - there is no JSON encoding.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select script:
Step 5 Click in the script version field and select the script/version that you want to apply for this field transformation - for example:
All scripts and versions are available for selection. If you choose a script/version that is not currently deployed, it will be deployed automatically.
Step 6 Accept your changes:
...then save the transformation:
Step 7 Now you can select/update the target field and then the mapping in the usual way.
The null value transform function is used to replace the value of a source field with null.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select null value from the other category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
Convert values using a .
Apply a field-level . Note that a script will time out if it runs for more than 120 seconds.
Specifier | Summary |
---|---|
Specifier | Summary |
---|---|
Specifier | Summary |
---|---|
Specifier | Summary |
---|---|
In our example, our source data is coming in via a manual payload so are defining the payload field manually - if you're using a to receive data, you'll be able to select the required field from the associated schema for your connection.
Step 9 Go to .
All source fields that were added for this mapping in will be available for selection here.
d
Day of the month with leading zeros (01 to 31).
j
Day of the month without leading zeros (1 to 31).
D
A textual representation of a day in three letters (Mon to Sun)
l
A full textual representation of the day of the week (Monday to Sunday)
m
Numeric representation of a month with leading zeros (01 to 12).
n
Numeric representation of a month without leading zeros (1 to 12).
M
A textual representation of a month in three letters (Jan to Dec)
F
A full textual representation of a month (January to December).
Y
Four-digit representation of the year (e.g. 2023).
y
Two-digit representation of the year (e.g. 23).
H
Hour in 24-hour format with leading zeros (00 to 23).
i
Minutes with leading zeros (00 to 59).
s
Seconds with leading zeros (00 to 59).
a
Lowercase Ante meridiem (am) or Post meridiem (pm).
A
Uppercase Ante meridiem (AM) or Post meridiem (PM).
The does not contain one of many transform function is used to match string values against a (comma delimited) list of values that you want to exclude.
For details, please see: Using contains one of many or does not contain one of many for string filters.
The cache lookup transform function is used to lookup and use data from a cache created earlier in the flow.
If caches have been added to your current process flow or company-level caches have been added for use in any other process flows, you can reference these in field mapping transformations.
For details please see Referencing a cache in mapping transformations in our cache section.
The null to string transform function is used to convert incoming null
values to an empty string. For example:
...is converted to:
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select null to string from the other category:
Step 5 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The concatenate transform function is used to join the values for two or more source fields (using a given joining character) and then map the output of this transformation to a destination field.
For example, you might have a source system that captures the first name
and last name
for customer records, and then a destination system that expects this information in a single name
field.
In the instructions below, we'll step through the scenario mentioned above where our incoming payload includes the first name
and last name
for customer records, but our destination system expects this information in a single full_name
field. The steps required are detailed in two stages:
To begin, we need to update/add the required mapping row so that it includes all source fields that need to be joined and then pushed to the specified destination.
Step 1 In your process flow, access settings for your map shape:
Step 2
Find (or add) the mapping row which requires a concatenate transformation. In the example below, we have a row that's currently set to map the source first name
field into the destination full name
field:
Step 3 On the source side of the mapping row, we need to add all the fields that need to be joined. To do this, click the 'pencil' icon associated with the existing source field:
Step 4 Details for the selected field are shown - click the add source field button:
Step 5 Click the 'pencil' icon associated with the new source field:
Step 6 Move down and update the display name and payload fields for the second source field that you want to join - for example:
In our example, our source data is coming in via a manual payload so are defining the payload field manually - if you're using a connection shape to receive data, you'll be able to select the required field from the associated schema for your connection.
Step 7 Accept these changes to exit back to your mapping rows - notice that there are now two source fields associated with the row you updated:
Step 8 Repeat steps 3 to 7 to add any more source fields that you need to join.
Step 9 Go to stage 2.
With all required source fields defined for our mapping row, we can add a concatenate transform function to join the values for these fields.
Step 1 Select the add transform button for the required mapping rule - for example:
Step 2 Click the add transform button:
Step 3 Click in the name field and select concatenate from the string section in the list of transform functions:
...concatenate options are displayed:
Step 4 In the join character field, enter the character that you want to join each of your source fields - for example, a hyphen or a space:
Step 5 Click the add field button:
Step 6 Click in source fields and select the first source field to be joined:
All source fields that were added for this mapping in stage 1 will be available for selection here.
Step 7 Accept your changes.
Step 8 Click the add field button again:
...and add the next source field to be joined - for example:
Step 9 Accept your changes.
Step 10 Repeat steps 8 and 9 to add any more source fields to be joined. Each time you accept a new source field you'll see the sequence that they will be processed when this transform function runs - for example:
Fields are joined in the sequence that they are added here.
Step 11 Having added all required source fields to be joined, accept changes:
...then save the function:
Step 12 Ensure that the target field for this mapping row is set as required, then save the map shape. Next time the process flow runs, the given source fields for this mapping row will be joined and then that value is pushed to the target. The example below shows an incoming payload before and after the concatenate transformation is applied:
The null to string transform function is used to convert incoming null
values to an empty string. For example:
...is converted to:
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select null to zero from the other category:
Step 5 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The cast to float transform function is used to change the data type associated with a source field from string
to float
.
A float is a type of number which uses a floating point to represent a decimal or fractional value. They are typically used for very large or very small values where there are a lot of numbers after the point - for example: 5.3333 or 0.0001.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select cast to float from the string category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The cast to number transform function is used to change the data type associated with a source field from string
to number
. For example, you might have an id
field in a source system that's stored as number
value, but your destination system expects the id
to be a string
.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select cast to number from the string category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The custom string transform function is used to map a given string to a target field. This string can be static, or you can reference flow variables and cached data.
If you've added/updated a map shape before, you'll be used to selecting a source field and a target field. However, when a custom string transformation is used we don't select a source field - the custom string transformation is our data source.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select custom string:
Step 5 Move down to the custom string field and enter your required text or variables - for example:
For more information about referencing flow variables in a custom string, please see our Referencing flow variables in field mapping transformations page. For more information about referencing cached data in a custom string, please see our Referencing a cache in mapping transformations page.
Step 6 Accept your changes:
...then save the transformation:
Step 7 Now you can select a target field in the usual way - for example:
...then:
...then:
Step 8 Once your mapping is complete, the row should be displayed without a source field - for example:
From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the custom string (or associated values from variables) will be mapped to the given target field.
The replace transform function is used to replace an existing source string value with either:
An alternative string value
An empty value
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select replace from the string category:
Step 5 Update search and replace fields with your required values:
For the replace field, you can enter another string or leave the field blank to replace the source with an empty value.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The JSON encode transform function is encode incoming values as JSON. For example, you might have product data in a string:
...and need to encode the values for pushing to the destination system:
Although this is listed as a string type transform, in fact any data type can be encoded.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select JSON encode from the string category:
Step 5 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes:
Step 8 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The first word transform function is used to extract the first word of an incoming string value, based on a user-defined delimiter. For example, you might have product data in a string:
...and need to extract just the last item in the string for pushing to the destination system:
In this case, items in our source string value are delimited with a comma, so we can use this to determine the first word. The transform checks incoming string values and determines the 'first word' to be the word after the first occurrence of the given delimiter.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select first word from the string category:
Step 5 In the delimiter field, enter the character that delimits elements in the string:
If you use any of the following characters, they should be escaped:
.
+
*
?
^
$
(
)
[
]
{
}
|
\
/
For example, a delimiter of *
would be entered as:
\*
A space is a valid character.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice):
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The last word transform function is used to extract the last word of an incoming string value, based on a user-defined delimiter. For example, you might have product data in a string:
...and need to extract just the last item in the string for pushing to the destination system:
In this case, items in our source string value are delimited with a comma, so we can use this to determine the last word. The transform checks incoming string values and determines the 'last word' to be the word after the last occurrence of the given delimiter.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select last word from the string category:
Step 5 In the delimiter field, enter the character that delimits elements in the string:
If you use any of the following characters, they should be escaped:
.
+
*
?
^
$
(
)
[
]
{
}
|
\
/
For example, a delimiter of *
would be entered as:
\*
A space is a valid character.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes:
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
The pad transform function is used to pad an existing string of characters to a given length, using a given character. You can apply padding to the left (i.e. immediately before the existing string), to the right (i.e. immediately after the existing string), or both (immediately before and after, as equally as possible).
The payload item below contains a string that's 8 characters long:
If we apply padding to a length of 20
using a *
character to the right
, the result would be:
Here, we have an extra 12 * characters to the right, giving a string length of 20. However, if we apply the *
character to both
, the result would be:
Now the padding is applied with 6 characters to the left of the original string and 6 characters to the right.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform button for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select pad from the string section:
Step 5
Click in the direction
field and select where you would like padding to be applied:
You can apply padding to the left
(i.e. immediately before the existing string), to the right
(i.e. immediately after the existing string), or both
(immediately before and after, as equally as possible).
Step 6
In the length
field, specify the number of characters that you'd like the final (i.e. transformed) string to be - for example:
Step 7
In the pad character
field, specify the character that you'd like to use for padding - for example:
If you want padding to be applied with spaces, press the space bar once in this field.
Step 8 Click the add field button:
Step 6 Click in source fields and select the source field to be used for this transform:
Step 7 Accept your changes (twice).
Step 8 Save the transform.
The split string transform function is used to split elements of a string into an array, with a user-defined delimiter. For example, you might have product data in a string:
...and need to convert these items to an array before pushing to the destination system:
In this case, items in our source string value are delimited with a comma, so we can use this to determine where each split occurs. The transform checks incoming string values and determines each array item to be the word after each comma.
Step 1 In your process flow, access settings for your map shape:
Step 2 Select the add transform icon for the required mapping rule - for example:
Step 3 Click the add transform button:
Step 4 Click in the name field to access a list of all available transform functions, then select split from the string category:
Step 5 In the delimiter field, enter the character that delimits elements in the string:
If you use any of the following characters, they should be escaped:
.
+
*
?
^
$
(
)
[
]
{
}
|
\
/
For example, a delimiter of *
would be entered as:
\*
A space is a valid character.
Step 6 Click the add field button:
Step 7 Click in source fields and select the source field to be used for this transform:
Step 8 Accept your changes (twice).
Step 9 Save the transform. You'll notice that the transform icon is updated to indicate that a transform function is present for the mapping row - for example:
You need to map an array within a payload but also include one of the 'parent' fields - for example, map the following:
...to:
To achieve this, the target field must be defined with double *.
characters in the data path. For example:
If you don't do this, and just enter the standard single *.
characters - for example:
....the output will be a flat object - for example: