Filter shape

Introduction

When a connector is built, default filters can be applied at the API level, so when a process flow connection shape pulls data, the payload received is has already been refined.

However, there may be times where you want to apply additional filters to a payload that's been pulled via a connection shape - for example, if the API for a connector does not support particular filters that you need.

The filter shape works with a source payload. As such, it should be placed AFTER a connection shape in process flows.

Need to know

When specifying a filter value, the maximum number of characters is 1024.

Accessing filter shape settings

To view/update the settings for an existing filter shape, click the associated 'cog' icon:

This opens the options panel - for example:

Configuring settings for a new filter shape

Follow the steps below to configure a filter shape.

Step 1 Select a source integration and endpoint to determine where the incoming payload to be filtered originates.

Step 2 Click the add new filter button:

Step 3 Filter settings are displayed:

From here, you can select a field (from the data schema associated with the source endpoint selected in step 1) - for example:

Alternatively, you can toggle the manual input option to ON and add a manual path.

The manual data path field supports metadata variables.

Step 4 Use remaining operator, type and value options to define the required filter.

Presentation of the value field is dependant upon your selected type. For example, if the type field is set to specific date, you can pick a date for the value:

When defining a value, you can include payload, flow, and metadata variables.

Step 5 Use the keep matching? toggle option to choose how matched records should be treated:

Here:

• If the keep matching? option is toggled OFF, matched records are removed from the payload before it moves on down the flow for further processing.

• If the keep matching? option is toggled ON, matched records remain in the onward payload, and all non-matching records will be removed.

Step 6 Click the create button to confirm your settings.

Step 7 The filter is added to the filter shape - you can now add more filters if needed:

Don't forget that when a payload is running you can click the tick icon associated with any shape and view the payload at that point in time - this is a great way to check that your filter is refining data as expected.

Available filter types

When defining a filter, you can choose from the following types:

Type	Expected value
String	A text string - for example : blue Note that when a string type is selected, you can choose the regex operator and define a regex value. This provides greater flexibility if you can't achieve the desired results using standard operators. preg_match is used for pattern matching. For an example, see Using regex for string-type filters. You can also match a string-type field according to whether or not the value is one of many items in a given list. For more information, see Using contains one of many or does not contain one of many for string filters.
String length
Number	A number - for example: 2
Specific date	A day, month and year, selected from a date picker.
Dynamic date	Specify a date/time which is relative to a +/- number of units (seconds, minutes, hours, days, months, years). For example: It's important to be aware that relative date/time variables are affected by the use queued at time process flow setting.
Boolean
Null comparison	A field is null if it has no defined value. Choose whether the give data path is null or not null. For example:
Variable	Designed specifically for cases where you are comparing a variable value as the filter comparison. When selected, a value type field is displayed and the expected type for your variable can be selected. This ensures that a true comparison can be made. Note that if you are filtering on multiple variables, the value type field should be set to None.

Related information

• Using regex for string-type filters

• Using contains one of many or does not contain one of many for string filters