Process flow settings
Last updated
Last updated
Process flow settings are used to manage settings and behaviour for the process flow as a whole:
The ability to update some settings will vary, depending on the version status. For example, you can't add flow variables to a deployed version.
Process flow settings are summarised below.
1
Process flow name & description
The name displayed for this process flow throughout the system. Optionally, you can include a description.
2
Queue priority
All process flow run jobs are added to your queue and, by default, are picked for processing when a slot becomes available - i.e. they have a regular priority.
If required, you can use this dropdown field to select the priority in which runs for this process flow are picked from your queue. Choose from:
Highest. This job is picked from the queue before anything else. If more than one job is queued with this priority, the one queued first is picked.
High. This job is picked from the queue AFTER any highest priority jobs are cleared.
Regular. This job is picked from the queue at the first opportunity AFTER any high and highest priority runs are cleared. This is the default setting.
Low. This job is picked from the queue AFTER any regular priority jobs are cleared.
Lowest. This job is picked from the queue AFTER any low priority jobs are cleared.
4
Use queued time
Some process flow steps (connectors, filters, set variables, transforms, etc.) can be configured to use dynamic/relative dates, where the date is relative to the time that the variable is used in the process flow.
With the introduction of queues for process flow runs, all scheduled process flows are added to a queue when they are initialised. This means that the time a flow is initialised is not the same as the time the flow actually runs - sometimes the run will be almost instant, but in busier periods there may be some minutes between starting and running a flow.
To prevent cases where filtered records are omitted because they were added between the time a flow was initialised and the time it left the queue to run, the use queued time process flow setting can be used. This allows you to choose whether any relative dates should be based on:
the time that the variable is reached in the flow (i.e the current time)
the time that the process flow was queued
This option defaults to true for all new process flows.
Example
To find all records created in the last 2 hours, a relative date variable is configured as:
At 12:00 the process flow enters the queue. At this point, the value of this variable would be: 10:00
At 14:00 the process flow leaves the queue and runs. At this point, the value of this variable would be: 12:00
So:
With the use queued at time option set to ON, we take the time the flow entered the queue as our base for the relative date variable, so we'd retrieve all records created after 10:00.
With the use queued at time option set to OFF, we take the current time as our base for the relative date variable, so we'd retrieve all records created after 12:00.
5
Remove failed payloads
Toggle this option on if you want to remove payloads that would otherwise cause this flow to fail. This is useful if multiple payloads pass through a process flow (for example, if received data is paginated or batched via a flow control shape) and one or more of these includes a data issue.
With this option switched on, the failed payload is removed and the flow continues. If the process flow completes successfully, its status is set to partial success.
6
Production flow
If this option is toggled ON, Patchworks will receive alerts if a run fails for this flow, which may be analysed to understand trends and areas for future enhancements.
7
Labels
View and update labels associated with this process flow. You can remove an existing label, apply labels from the dropdown list, or create a new label.
8
Email failure notifications
Use the dropdown list to select a notification group to receive an email alert if a process flow run fails.
10
Variables
Define flow variables and then reference these values throughout the entire process flow.
11
Versions
All existing versions of a process flow are displayed. From here you can select any version to view the flow at that point in time. You can also choose to deploy the currently selected draft or inactive version, and to duplicate any version of a process flow.
If your process includes a run process flow shape, note that any 'sub flows' do NOT inherit the queue priority from the 'calling' flow. You should set the priority for these as required.
Failed payloads can be viewed in real-time run logs, via the failed payloads tab. These payloads can be downloaded from run logs:
