When a process flow is running, the logs panel provides real-time information about the progress of each step:
This is a great way to monitor what's happening during each step of the flow.
As a flow progresses, click the 'tick' icon for a shape to view information for that step - for example:
You can drag this panel wherever you like in the canvas. If you prefer to work with a larger window, click the 'pop-out' icon:
Details are provided over a series of tabs:
The logs tab provides a commentary about the status of the process flow as it progresses through each step - for example:
The payloads tab shows the payload for the selected step in the process flow - this means you can see exactly what's happened to your data at every stage. For example:
If a shape generates multiple payloads (for example, where a flow control shape batches data into smaller payloads), you can choose which one to view - for example:
If a payload is too large to display it is trimmed and a download payload option is available for offline viewing.
The requests tab shows details of the API request made for the selected shape - for example:
Here you can see the API request URL
, method
, body
content and headers
. If multiple requests are made (for example, if a connection fails the first time) you can use the request dropdown list to view each one.
Remember that every process flow shape is driven by an API request - the information here shows what's going on behind the scenes.
The responses tab shows details of the API response received for the selected shape - for example:
Here you can see the API response status
, body
content and headers
. If multiple requests were made (for example, if a connection failed the first time) you can use the response dropdown list to view each one.
Once you start syncing data with process flows, we provide the tools to make sure that you're in the driver's seat when it comes to knowing that everything is running as expected and - in the event of any issues - understanding when/where a problem occurs so you can make informed decisions to resolve it.
Your toolkit includes:
Real-time logs for immediate feedback as a process flow runs. Follow the progress of every step in a process flow with success/failure details, payloads, and request/response information for every shape.
Retrospective run logs are retained for one month so you can access success/failure details, payloads, request and response information retrospectively.
Email alerts for failed runs so you're not constantly checking logs for assurance that your process flows are running as expected.
The run logs page is used to access detailed logs and payloads for active and previous process flow runs and also your run queue:
Run logs are retained for 7 days.
Payloads are retained for 72 hours.
Payloads are stored in AWS S3 (eu-west-2 London).
Payloads are encrypted during any pull/push operations (HTTPS, TLS 1.2/1.3).
To access retrospective run logs, follow the steps below.
Step 1 Log into the Patchworks dashboard and select run logs from the left-hand navigation menu:
Step 2 Select the active tab for active and previous runs or the queue tab for pending runs:
The active tab displays active and previous process flow runs - these are listed chronologically based on the run start time:
At a glance, you can see the outcome of a run, the start date/time, the duration, and how the run was triggered. Failed runs are shown with an expandable summary of why the run failed, and a view logs option is available for each entry so that you can access detailed logs and payloads.
If a process flow is running it is displayed with a 'stop' button that you can use to stop the run.
Run logs are retained for 7 days.
Payloads are retained for 72 hours.
Payloads are stored in AWS S3 (eu-west-2 London).
Payloads are encrypted during pull/push operations (HTTPS, TLS 1.2/1.3).
By default, logs are displayed for the last seven days. You can change this using the associated date filter - here, you can clear all filters to show all available logs, or choose a new date/time range:
Use filters to refine the list based on the value associated with a particular column. Currently, you can filter logs by:
Date
Status
Trigger (schedule, manual, API, etc.)
To define a filter, click the filter icon associated with a column - for example:
...then make selections from the options provided - for example, the date filter displays date/time pickers:
The status of a run will be one of the following:
Click the hamburger icon associated with a run log entry to access detailed logs and payloads:
Here, you can scroll down the page to view information for each step of the process flow:
Click the 'eye' icon to view the associated payload(s) for a log item:
Any associated payloads are available for selection:
Payloads are available for 72 hours after the process flow has run.
If a payload is too large to display it is trimmed and a download payload option is available for offline viewing.
For information about downloading run logs, please see our Downloading run logs page.
Run logs can contain a lot of information and there may be times when you just want to quickly see the reason a run has failed.
When a process flow run is initialised but fails somewhere in the run, its status is set to failed and an expandable arrow is shown to the left of the entry - click this arrow to view a summary of why the failure occurred. For example:
If a run entry shows as failed, it is displayed with a retry button - for example:
You can click this to try the run again. This run job is then added to the queue, to be processed as soon as possible.
When this option is used, the flow version that was set originally is used. If you want to run an updated version of the process flow, you can choose to edit the flow and initialise a run manually instead.
If a process flow includes references to variables, be aware that any values for those variables may not be the same as they would have been if the failed flow had been successful.
When a job is triggered to run (either manually, or via a trigger/webhook/inbound API call), the run job is placed in a queue for processing. You can view all of your pending runs from the queue tab - entries are listed chronologically based on the time that the job was added to the queue:
At a glance, you can see the run id, process flow version id, how the run was triggered, the status, and the date/time that the run job was created/updated.
As soon as a pending job starts running, the entry is removed from this table and moves to the active run logs section.
The created date/time is the date and time that the run was added to the queue. Since we are only considering pending items, the created and updated date/time will always be the same.
The status of a queue entry will always be set to pending.
There is a limit on the number of process flow runs that can be started per minute. This limit varies according to your subscription tier - please see Core subscription tiers for information.
Status | Summary |
---|---|
Success
The process run completed without errors.
Failed
The process run did not complete due to errors.
Stopped
The process flow was stopped manually.
Running
The process flow is currently running.
You can configure a process flow to trigger email notifications in the event that a process flow runs and fails. This is a two-stage process:
If you're not sure how to approach a process flow that's failed, see our Process flow run failures troubleshooting guide.
Email notifications for a failed process flow run are only sent if:
the process flow is enabled and deployed
the process flow is initiated via the trigger shape (i.e. not for manual runs)
Having added email recipients to a notification group, you should access settings for the required process flow(s) and apply this group to the email failure notifications section. To do this, follow the steps below.
Step 1 Access the process flow that you want to update.
Step 2 Click the 'cog' icon in the canvas toolbar to access general settings for this process flow:
Step 3 From the settings panel, click in the email failure notifications field and select the appropriate group to receive an email notification in the event that this process flow runs and fails:
All defined notification groups are available for selection. If you to add a new group or check recipients in an existing group, please check our Notification groups page.
Step 4 Save changes.
If required, you can download run logs for a completed process flow run in CSV format, for use outside of Patchworks.
Run log exports are completed in CSV format. The exported file includes two columns with level
and message
headers. For example:
To download run logs for a specific process flow run, follow the steps below.
Step 1 Log into the Patchworks dashboard, then select the process flows | run logs option - existing run logs are displayed.
Step 2 Click the download option associated with the required run - for example:
Step 3 The download job is added to a queue and a confirmation message is displayed:
Step 4 When your download is ready, you'll receive an email which includes a link to retrieve the file from the file downloads page. If you can't/don't want to use this link, you can access this page manually by selecting the settings option:
...followed by the file downloads option:
Step 5 On the file downloads page, you'll find any exports that have been completed for your company profile in the last hour. Click the download button for your job:
This list may include exports from different parts of the dashboard, not just run logs (for example, de-dupe and cross-reference lookup data exports are added here).
Step 6 The associated CSV file is saved to the default downloads folder for your browser.
Download files are cleared after one hour. If you don't manage to download your file within this time, don't worry - just download the logs again to create a new one.