There may be times when you need to define a filter based on incoming data matching one of many given values. Conversely, you might want to define a filter based on incoming data NOT matching one of many given values. This can be achieved using the following operators in string-type filters:
Using these operators, you can specify a comma-separated list of values that a record must have/not have in a string-type field, to be a match.
This information applies wherever filters are available, not just the filter shape.
These operators are designed to work with string-type fields only.
The contains one of many operator is used to match incoming records if the value of a given field DOES match any item from a provided (comma delimited) list of values. For example, consider the following payload of customer records:
Suppose you only want to process customer records with a European country code in the country field (which is a string type field).
You can add a filter for the country field and select the contains one of many operator - then provide a comma-separated list of acceptable country codes as the value:
The resulting payload would only include records where the country field includes one of the specified values - i.e.:
The does not contain one of many operator is used to match incoming records if the value of a given field DOES NOT match any items from a provided (comma delimited) list of values. For example, consider the following payload of customer records:
Suppose you only want to process customer records that do NOT have US or AU in the country field (which is a string type field).
You can add a filter for the country field and select the does not contain one of many operator - then provide a comma-separated list of unacceptable country codes as the value:
The resulting payload would only include records where the country field does NOT include one of the specified values - i.e.:
When defining your 'many values' list as the value for a contains one of many or a does not contain one of many filter, there are a couple of things to keep in mind:
It's important that your 'many values' are specified as a comma-separated list - so in our example:
...will match as required, but:
...will NOT match as required.
Any spaces included in your 'many values' list ARE considered when matching. For example, consider our original payload:
Suppose we are using the contains one of many operator to match all European countries and the value field is defined as below:
Notice that the final IE list item is preceded by a space. This means that our filter will only match the IE country code if it's preceded by a space in the payload, so the output would be:
Our IE record (line 8 in the payload) isn't matched because there's no space before the country code in the country field.
When a connector is built, default filters can be applied at the API level, so when a process flow pulls data, the payload received is has already been refined.
However, there may be times where you want to apply additional filters to a payload that's been pulled via a - for example, if the API for a connector does not support particular filters that you need.
The filter shape works with a source payload. As such, it should be placed AFTER a in process flows.
When specifying a filter value, the maximum number of characters is 1024.
To view/update the settings for an existing filter shape, click the associated 'cog' icon:
Follow the steps below to configure a filter shape.
Step 1 Select a source integration and endpoint to determine where the incoming payload to be filtered originates.
Step 2 Click the add new filter button:
Step 3 Filter settings are displayed:
From here, you can select a field (from the data schema associated with the source endpoint selected in step 1) - for example:
Alternatively, you can toggle the manual input option to ON and add a manual path.
Step 4 Use remaining operator, type and value options to define the required filter.
Step 5 Use the keep matching? toggle option to choose how matched records should be treated:
Here:
If the keep matching? option is toggled OFF, matched records are removed from the payload before it moves on down the flow for further processing.
If the keep matching? option is toggled ON, matched records remain in the onward payload, and all non-matching records will be removed.
Step 6 Click the create button to confirm your settings.
Step 7 The filter is added to the filter shape - you can now add more filters if needed:
When defining a filter, you can choose from the following types:
When adding filters for a string-type field, there may be times when standard operators can't achieve what you need. For example, when defining multiple filter conditions, the default condition is AND - so ALL specified filters must be met for a match. But what if you need an OR condition?
For more complex filtering requirements, regex can be used.
This information applies wherever filters are available, not just the filter shape.
Let's take the following payload:
Suppose we want to retrieve any items where the value of the fruit field contains peaches OR apples. We might be tempted to add two string-type filters in the filter shape:
However, this wouldn't return any matches because we'd be looking for any records where peaches AND apples are present. Instead, we can define one string-type filter with regex:
This opens the - for example:
The manual data path field supports .
Presentation of the value field is dependant upon your selected . For example, if the type field is set to specific date, you can pick a date for the value:
When defining a value, you can include , , and variables.
Don't forget that when a payload is running you can - this is a great way to check that your filter is refining data as expected.
| Type | Expected value |
|---|
String | A text string - for example : |
String length |
Number | A number - for example: |
Specific date | A day, month and year, selected from a date picker. |
Dynamic date | Specify a date/time which is relative to a +/- number of units (seconds, minutes, hours, days, months, years). For example: |
Boolean |
Null comparison | A field is |
Variable | Designed specifically for cases where you are comparing a variable value as the filter comparison. When selected, a |
This will return the correct payload - i.e. any records where peaches AND apples are present:
Note that when a string type is selected, you can choose the regex operator and define a regex value. This provides greater flexibility if you can't achieve the desired results using standard operators. preg_match is used for pattern matching. For an example, see .
You can also match a string-type field according to whether or not the value is one of many items in a given list. For more information, see .
A number which represents the expected string length for the received payload. Here, the 'payload' might refer to a targeted field within the incoming payload, or the entire payload.
For example, if you want to ensure that an objectId field is never empty, you would define a filter for objectId greater than 0:
In this case, toggling the keep matching option to OFF means that the ongoing payload will include only items where this field is not empty. Conversely, toggle this option on if you want to pass on empty payloads for any reason.
You can use the same principle to check for empty payloads (as opposed to a specific field). In this case you would define a filter for * greater than 0:
It's important to be aware that relative date/time variables are affected by the .
A true or false value. For example, if you only want to consider items where an itemRef field is set to true, you would define a filter for itemRef equals true:
