1 of 7

Dynamic variables

Introduction

As you work with shapes in process flows, you'll be used to updating shape settings with required values. Mostly, you'll define static values - for example, selecting or entering a data item to be used as a de-dupe key, entering a key name for an add to cache shape - there are dozens of settings that you might configure when building process flows.

However, there may be times where the value of a field can't be defined as a static value because it needs to be resolved dynamically, based on data received from the incoming payload and/or defined for the process flow as a whole. This can be achieved using:

Where can dynamic variables be used?

Lots of our process flow shapes include settings where you can enter a static value, or provide a variable/parameter to be resolved dynamically. Please refer to our for specific guidance for each shape.

Payload variables

Introduction

There may be times when you need to define variables or parameters for a process flow shape which resolve dynamically, based on given values from the incoming payload.

For example, you might have a list of customer IDs coming in from a source (for example, an inbound API job), and need to match these IDs with payload data for a subsequent connection shape, in order to create customer records. This would be achieved using a specific when defining variables in the .

Payload variable syntax

To pass in a variable or parameter value from the incoming payload, use the syntax shown below:

...where schema notation should be replaced with the relevant notation path for the required field in the incoming schema. A payload variable can be defined on its own, or combined with static text. For example:

If necessary, you can combine payload variables with metadata and/or flow variables - for example:

Supported

The [[payload]] variable supports non-JSON payload data types - for example, raw text, CSV, XML - whatever you pass in will be output.

Payload variable examples

To show how this works in principle, some examples are detailed below:

However, it's important to note that the required settings will depend on the data schemas used for your selected connection endpoints.

Example: single item

At the most basic level, your incoming data might contain items which aren't nested - for example:

In this case, our variable would be defined as:

...as shown below:

The result for our example would be:

Example: all items in an array

An incoming payload might contain the required element in an array and you want to target all items within it - for example:

In this case, we can define a variable to target the required array. For example:

...will produce a comma separated list of associated customerID values. The result for our example would be:

Example: single item in an array

An incoming payload might contain the required element in an array and you only want to target a single item - for example:

In this case, we can define a variable to target the required array item. For example:

...will target the first item in the array and return the associated customerID value. The result for our example would be:

Whereas:

...will target the second item in the array and return the customerID value. The result for our example would be:

Example: batched items from an array

Let's take our example below:

This will result in three payloads, each with a single customerID and name. For example:

In the following connection step (where the variable/parameter is defined), we can add our payload syntax for the variable/parameter. For our example this would be:

...as shown below:

Here, we need to specify the * because each of our batched payloads is wrapped in an array. For each payload generated from our example, the result is that the associated customerID is taken as the variable value.

Metadata variables

Introduction

Payload metadata can be added by , and also by the shape.

There may be times when you need to define variables or parameters for a process flow step using values from the incoming payload metadata. This can be achieved using a specific payload syntax in the process flow connection shape.

Need to know

Payload metadata cannot exceed 10240 bytes. Exceeding this limit will cause the process flow to fail on the associated step.

Metadata variable syntax

Payload metadata can be accessed via a variable or parameter field using the syntax below:

For example:

When defining variables you can 'mix and match' , metadata and variables. For example:

When a payload is passed to a 'child' process flow via the , meta variables are included.

Flow variables

Introduction

If your process flow includes a that's configured for an endpoint where a variable can be entered, you'll probably be used to entering a static value to be applied for that step - for example:

You might also be familiar with . However, flow variables provide another level of flexibility.

Flow variables provide the ability to define variables at the process flow level, and then reference these values throughout the entire process flow. You set a flow variable once, and it is applied throughout the entire process flow, wherever it is referenced.

When flow variables are modified - either manually or via a script - those updates are applied anywhere in the process flow where they are referenced, automatically.

You can also reference flow variables in (which means you can manipulate these values however you need) and also in .

Need to know

Before you start working with flow variables, there are a couple of important points to understand regarding and .

Process flow versions

Flow variables are version-specific. For example, if you add flow variables to the current draft version and later restore an inactive version to draft, any defined flow variables won't be present. So, make sure you're updating the correct version of a process flow. For more information, please see our page.

Update persistence

If you update a flow variable , those updates persist for the duration of the flow run. Once the process flow has been completed, default values are restored.

Using flow variables

Please see the following pages:

Referencing flow variables in a process flow

Introduction

The steps required to reference flow variables in a process flow can be summarised in two stages:

Stage 1: Define flow variables that can be referenced
Stage 2: Add references to defined flow variables in process flow steps

Stage 1: Define flow variables that can be used

Any flow variables that you want to reference from process flow shapes should be added as variables within the process flow settings. To do this, follow the steps below.

Step 1 Access the process flow that you want to update and make sure that you're switched to the required version.

Typically, you would update the draft version, then deploy changes when you are ready.

Step 2 Select settings (the cog icon) from the actions bar:

Step 3 Look for the variables section in the flow settings panel - for example:

Step 4 Click the add new variable button:

Step 5 In the name field, enter the name (i.e. the API parameter name) of this variable. For our example, the variable is named customerID:

Step 6 Click in the select a type field and select the data type for this variable:

Step 7 Enter the required value to be used wherever this variable is found in the process flow - for example:

Step 8 Add all required flow variables in the same way, then save changes.

Stage 2: Add references to defined flow variables

Having defined your required flow variables, they can be referenced from process flow connection shapes, wherever a variable field is present. So, if your connection shape is set to use an endpoint which requires/allows variables to be applied, you will see corresponding variable fields in the connection shape settings.

The example below shows how this works. A GET single order endpoint has been configured to expect a customerID variable, and then how this variable is surfaced in connection shape settings when this endpoint is used:

When defining variable values for a connection shape, you can enter a static value, or obtain values dynamically from a payload, or reference an existing flow variable. The steps below show how to reference a flow variable.

You can also reference flow variables in custom scripts (which means you can manipulate these values however you need) and also in field mapping transformations.

Step 1 In your process flow, access settings for the connection shape that you want to update with a flow variable:

Step 2 Look for the variables section in the settings panel - for example:

To use a flow variable here, the expected variable must correlate with a variable that you added in stage 1. Notice that the example above is expecting a Customer ID variable, which correlates with the customerID flow variable that we added in step 5 of stage 1.

Step 3 Use the syntax below to reference a flow variable:

{{flow.variables.variable}}

...where the variable element should be replaced with the name of the flow variable defined in process flow settings (stage 1). Using our example, this would be:

{{flow.variables.customerID}}

Step 4 Save changes. Now when this process flow runs, the value defined in process flow settings will be passed in for this variable.

Flow variable values can also be updated by custom scripts. For further information please see Referencing flow variables in custom scripts.

Referencing flow variables in custom scripts

Introduction

If required, you can reference (and therefore manipulate) flow variables in custom scripts. The possibilities here are only as limited as your development expertise however, a simple example might be where you want to generate a running count of order lines, to be output to a total field in a target system.

To achieve this, you would:

Add a flow variable named running_total to your process flow settings.
Write a custom script which loops over each received order line and updates the running_total variable as it goes.
Add the custom script to your process flow via a script shape.
Add a map shape to your process flow and include a rule which maps a custom string transform for {{flow.variables.running_total}}, to the total field in the target system.

Flow variable syntax for custom scripts

To reference flow variables in a custom script, the required syntax is as follows:

PHP 8.1

['flow']['variables']['variableName']

Javascript Node 18

flow.variables.variableName

C# 8

["flow"]["variables"]["variableName"]

Python 3

['flow']['variables']['variableName']

Ruby 3

['flow']['variables']['variableName']

In all cases, the variableName element should be replaced with the actual flow variable name. For example:

PHP 8.1

['flow']['variables']['running_total']

Example

The example script below takes a flow variable named customerID and sets the value to 1234567:

PHP 8.1

<?php

/**
 * Handler function.
 *
 * @param array $data [
 *      'payload'   => (string|null) the payload as a string|null
 *      'variables' => (array[string]string) any variables as key/value
 *      'meta'      => (array[string]string) any meta as key/value
 *    ]
 */

function handle($data)
{
    $data['flow']['variables']['customerID'] = 1234567;

    return $data;
}

So, wherever the customerID flow variable is referenced in a process flow, its value would be set to 1234567 when the process flow runs.

When you update flow variables via a script, those updates persist for the duration of the flow run. Once the process flow has completed, default values are restored.

Referencing flow & meta variables in mapping transformations

the Introduction

You can reference flow variables in field mapping transformations, using a custom string transformation in the map shape.

Most typically, the custom string transformation type is used to specify some custom text to be applied in a target field. For example, you might choose to add a particular sales rep's name to a reference field for all orders being synced.

However, if your process flow is using flow variables, you can reference these in custom string transformations, instead of static text. When the process flow runs, the flow variable value is mapped to your given target field. If you later decide to update flow variables (either manually or via a custom script), those updates are mapped next time the process flow runs.

The steps required to achieve this can be summarised in two stages:

Stage 1: Define flow variables that can be referenced
Stage 2: Add references to defined flow variables

Stage 1: Define flow variables that can be used

Any flow variables that you want to reference in field transformations should be added as variables within the process flow settings. To do this, follow the steps below.

Step 1 Access the process flow that you want to update and make sure that you've switched to the required version.

Typically, you would update the draft version, and then deploy changes when you are ready.

Step 2 Select settings (the cog icon) from the actions bar:

Step 3 Look for the variables section in the flow settings panel - for example:

Step 4 Click the add new variable button:

Step 5 In the name field, enter the name (i.e. the API parameter name) of this variable. For our example, the variable is named rep:

Step 6 Click in the select a type field and select the data type for this variable:

Step 7 Enter the required value to be used wherever this variable is found in the process flow - for example:

Step 8 Save changes.

Stage 2: Add references to defined flow variables

Having defined your required flow variables, they can be referenced in custom string field mapping transformations. Typically, this will be for scenarios where you want to map the output from a flow variable into a target field.

If you've added/updated a map shape before, you'll be used to selecting a source field and a target field. However, when we map a flow variable value to a target field, we don't select a source field - the custom string transformation is our data source.

Step 1 In your process flow, access settings for the map shape that you want to update with a flow variable:

Step 2 Click the add mapping rule option - for example:

Step 3 Click the add transform button:

Step 4 Click the add transform button:

Step 5 Click in the name field to access a list of all available transform functions, then select custom string:

Step 6 Move down to the custom string field and enter the following value:

{{flow.variables.variable}}

...where the variable element should be replaced with the name of the flow variable defined in process flow settings (stage 1) that you want to reference. Using our example, this would be:

{{flow.variables.rep}}

For example:

You can also reference metadata variables in the same way, using the meta syntax:

[[meta.unique_key]]

Step 7 Accept your changes:

...then save the transformation:

Step 8 Now you can select a target field in the usual way - for example:

...then:

Step 9 Once your mapping is complete, the row should be displayed without a source field - for example:

From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the value of this flow variable will be mapped to the target field.

Referencing flow variables in a process flow

Introduction

The steps required to reference flow variables in a process flow can be summarised in two stages:

Stage 1: Define flow variables that can be referenced
Stage 2: Add references to defined flow variables in process flow steps

Stage 1: Define flow variables that can be used

Any flow variables that you want to reference from process flow shapes should be added as variables within the process flow settings. To do this, follow the steps below.

Step 1 Access the process flow that you want to update and make sure that you're switched to the required version.

Typically, you would update the draft version, then deploy changes when you are ready.

Step 2 Select settings (the cog icon) from the actions bar:

Step 3 Look for the variables section in the flow settings panel - for example:

Step 4 Click the add new variable button:

Step 5 In the name field, enter the name (i.e. the API parameter name) of this variable. For our example, the variable is named customerID:

Step 6 Click in the select a type field and select the data type for this variable:

Step 7 Enter the required value to be used wherever this variable is found in the process flow - for example:

Step 8 Add all required flow variables in the same way, then save changes.

Stage 2: Add references to defined flow variables

You can also reference flow variables in custom scripts (which means you can manipulate these values however you need) and also in field mapping transformations.

Step 1 In your process flow, access settings for the connection shape that you want to update with a flow variable:

Step 2 Look for the variables section in the settings panel - for example:

Step 3 Use the syntax below to reference a flow variable:

{{flow.variables.variable}}

...where the variable element should be replaced with the name of the flow variable defined in process flow settings (stage 1). Using our example, this would be:

{{flow.variables.customerID}}

Step 4 Save changes. Now when this process flow runs, the value defined in process flow settings will be passed in for this variable.

Flow variable values can also be updated by custom scripts. For further information please see Referencing flow variables in custom scripts.

Payload variables

Introduction

There may be times when you need to define variables or parameters for a process flow shape which resolve dynamically, based on given values from the incoming payload.

Payload variable syntax

To pass in a variable or parameter value from the incoming payload, use the syntax shown below:

If necessary, you can combine payload variables with metadata and/or flow variables - for example:

Supported

The [[payload]] variable supports non-JSON payload data types - for example, raw text, CSV, XML - whatever you pass in will be output.

Payload variable examples

To show how this works in principle, some examples are detailed below:

However, it's important to note that the required settings will depend on the data schemas used for your selected connection endpoints.

Example: single item

At the most basic level, your incoming data might contain items which aren't nested - for example:

{
 "customerID":1000001
}

In this case, our variable would be defined as:

[[payload.customerID]]

...as shown below:

The result for our example would be:

{
 1000001
}

Example: all items in an array

An incoming payload might contain the required element in an array and you want to target all items within it - for example:

{
	"users": [{
			"customerID": "000001",
			"name": "Rita"
		},
		{
			"customerID": "000002",
			"name": "Bob"
		},
		{
			"customerID": "000003",
			"name": "Sue"
		}
	]
}

In this case, we can define a variable to target the required array. For example:

[[payload.*.customerID]]

...will produce a comma separated list of associated customerID values. The result for our example would be:

000001, 000002, 000003

Example: single item in an array

An incoming payload might contain the required element in an array and you only want to target a single item - for example:

{
	"users": [{
			"customerID": "000001",
			"name": "Rita"
		},
		{
			"customerID": "000002",
			"name": "Bob"
		},
		{
			"customerID": "000003",
			"name": "Sue"
		}
	]
}

In this case, we can define a variable to target the required array item. For example:

[[payload.users.0.customerID]]

...will target the first item in the array and return the associated customerID value. The result for our example would be:

Whereas:

[[payload.users.1.customerID]]

...will target the second item in the array and return the customerID value. The result for our example would be:

Example: batched items from an array

Let's take our example below:

{
	"users": [{
			"customerID": "000001",
			"name": "Rita"
		},
		{
			"customerID": "000002",
			"name": "Bob"
		},
		{
			"customerID": "000003",
			"name": "Sue"
		}
	]
}

We've seen how we can target , and how we can - but what if we wanted to target all array items individually?

In this case, we would add a shape to split the incoming payload into batches of 1 at the required data element level (in the case of our example this would ne users :

This will result in three payloads, each with a single customerID and name. For example:

[{"customerID":"000001","name":"Rita"}]

[{"customerID":"000002","name":"Bob"}]

[{"customerID":"000003","name":"Sue"}]

In the following connection step (where the variable/parameter is defined), we can add our payload syntax for the variable/parameter. For our example this would be:

[[payload.*.customerID]]

...as shown below:

Referencing flow & meta variables in mapping transformations

the Introduction

You can reference flow variables in field mapping transformations, using a custom string transformation in the map shape.

The steps required to achieve this can be summarised in two stages:

Stage 1: Define flow variables that can be referenced
Stage 2: Add references to defined flow variables

Stage 1: Define flow variables that can be used

Any flow variables that you want to reference in field transformations should be added as variables within the process flow settings. To do this, follow the steps below.

Step 1 Access the process flow that you want to update and make sure that you've switched to the required version.

Typically, you would update the draft version, and then deploy changes when you are ready.

Step 2 Select settings (the cog icon) from the actions bar:

Step 3 Look for the variables section in the flow settings panel - for example:

Step 4 Click the add new variable button:

Step 5 In the name field, enter the name (i.e. the API parameter name) of this variable. For our example, the variable is named rep:

Step 6 Click in the select a type field and select the data type for this variable:

Step 7 Enter the required value to be used wherever this variable is found in the process flow - for example:

Step 8 Save changes.

Stage 2: Add references to defined flow variables

Step 1 In your process flow, access settings for the map shape that you want to update with a flow variable:

Step 2 Click the add mapping rule option - for example:

Step 3 Click the add transform button:

Step 4 Click the add transform button:

Step 5 Click in the name field to access a list of all available transform functions, then select custom string:

Step 6 Move down to the custom string field and enter the following value:

{{flow.variables.variable}}

...where the variable element should be replaced with the name of the flow variable defined in process flow settings (stage 1) that you want to reference. Using our example, this would be:

{{flow.variables.rep}}

For example:

You can also reference metadata variables in the same way, using the meta syntax:

[[meta.unique_key]]

Step 7 Accept your changes:

...then save the transformation:

Step 8 Now you can select a target field in the usual way - for example:

...then:

Step 9 Once your mapping is complete, the row should be displayed without a source field - for example:

From here you can save changes or add more mapping rules as needed. Next time the process flow runs, the value of this flow variable will be mapped to the target field.