Introduction

Having accessed run logs and selected the view logs option for a given flow run, you can view:

• An insights summary with performance-related information for the process flow

• Detailed logs for each step in the flow

Insights summary

The insights summary is divided into three functional areas:

In the top, right-hand corner (1) you'll find additional options via an ellipses menu - these options will vary, depending on whether the run succeeded or failed. See Additional options for more information.

In the top panel (2), you can see basic details for the process flow (name, id, version and status, and how triggered) together with duration, CPU usage time, data used, and score.

The lower panel (3) shows every step (i.e. every shape) in the flow in processing sequence, with the following information:

Understanding scores

The score is a measure of how efficient/expensive (in terms of processing) your process flows are, based on the volume of data processed per second - 999 is the highest score.

Your score is based on all runs for all process flows associated with your company profile. This includes flows that are:

• Triggered by a schedule, webhook, or event

• Initialised by an API request

• Run manually

• Enabled or disabled when run

• Draft or deployed status when run

If your score is on the low side, it may be that your process flow is necessarily complex - including items such as scripts, transformations, flow control, caches, etc., has an impact on your score.

However, it's always worth checking a low-scored process flow further as there may be places where your process flows can be optimised (for example, does a flow include lots of mapping transformations that could be achieved in a single script?). A quick scan down the shapes list in the lower panel will show, at a glance, which shapes in your flow are taking longer to process or perhaps processing unusually high volumes of data.

If your score is low and you're satisfied that the process flow is built optimally, don't worry too much- scores are just there as a flag to indicate that it's worth checking the efficiency of your flow.

Detailed logs

In the lower panel of the insights summary, you can click any entry to access detailed logs:

In the top section, you can see a full history of what happened during this step - this will often display over multiple pages:

In the lower section, you can view and download any payloads associated with this step. Click the 'view' icon associated with any payload to see the contents:

...the payload is displayed:

Longer payloads are trimmed but you can use the download option to retrieve the full content:

Additional options

The ellipses icon in the top, right-hand corner provides access to additional options. These options vary, depending on whether the run failed or succeeded:

When a run is successful or partially successful, you can download run logs. If the run failed, you can download logs and also retry the run.

Introduction

The run logs page is used to access detailed logs and payloads for active and previous process flow runs and your run queue.

Accessing your run logs & queue page

To access the run logs & queue page, follow the steps below.

Step 1 Log into the Patchworks dashboard and select run logs from the left-hand navigation menu:

Step 2 Select the active tab for active and previous runs or the queue tab for pending runs:

Introduction

When a process flow run is initialised automatically (via a or a request), the run job is added to your queue.

(initialised from the canvas actions bar) and are NOT added to your queue:

How it works

The time that run jobs remain in the queue depends on various factors which are summarised below:

Points to note:

• Every 05 seconds, the system checks if any process flows are running:

  • If NO, it checks the queue and picks more flows to run - the number of flows picked will be the maximum allowed to meet your concurrent process flows limit.

  • If YES, it checks if our concurrent process flows limit has been reached. If it has, no more flows are picked from the queue. If it has not, the queue is checked and more flows are picked to run. The number of flows picked will be the maximum allowed to meet our concurrent process flows limit.

Viewing your run queue

You can view all of your pending runs from the queue tab - entries are listed chronologically based on the time that the job was added to the queue:

At a glance, you can see:

• Process flow name (hyperlinked so you can quickly access the flow)

• Trigger

• Queued at time

• Priority

From here you can:

Removing a specific run job

To remove a specific run job from your queue (so it doesn't run), click the associated 'bin' icon:

When prompted, confirm this action.

Removing all run jobs for a selected process flow

To remove all run jobs for a particular process flow (so none of them run):

Step 1 Select the required process flow from the dropdown list at the top of the page:

Step 2 Click the remove all for flow button:

When prompted, confirm this action.

Clearing all run jobs

To clear ALL pending run jobs and empty your queue, click the clear queue button:

When prompted, confirm this action.

Related information

Introduction

Having accessed run logs and selected the view logs (classic) option for a given flow run, you can view detailed logs and payloads for that run:

Here, you can scroll down the page to view information for each step of the process flow.

Viewing & downloading payloads

Click the 'eye' icon associated with the run that you want to access:

The log entry expands and further options are available. Click the 'eye' icon to show the payload:

Longer payloads are trimmed but you can use the download option to retrieve the full content:

Introduction

Having accessed the run logs & queue page, the active tab displays active and previous process flow runs - these are listed chronologically based on the run start time:

At a glance, you can see the outcome of a run, the start date/time, the duration, how the run was triggered and whether there were any warnings.

Failed runs are shown with an expandable summary of why the run failed, and you can also access additional options to access detailed logs, and more.

Need to know

• Run logs are retained for 7 days.

• Payloads are retained for 72 hours.

• Payloads are stored in AWS S3 (eu-west-2 London).

• Payloads are encrypted during pull/push operations (HTTPS, TLS 1.2/1.3).

• There is a limit on the number of process flow runs that can be started per minute. This limit varies according to your subscription tier - please see Core subscription tiers for information.

Searching run logs

If you're looking for a specific log, you can use the search field to search by flow run id or by process flow name:

Searching by flow run ID

If you want to find logs for a specific flow run id, type the full id into the search field - for example:

You must enter the id in full - partial elements are not matched.

A flow run id search is not case sensitive - entering 01j6errxr432r4k4dh41y5ttpc is the same as 01J6ERRXR432R4K4DH41Y5TTPC or 01J6eRRxR432r4K4DH41Y5ttPC.

Searching by flow name

If you want to find logs for a process flow that you only know by name, type any part of the name into the search field - for example:

You can enter as much or as little of the name as you like.

A flow name search is not case sensitive - entering CHECK is the same as check or Check.

The default view

By default, logs are displayed for the last seven days. You can change this using the associated date filter - here, you can clear all filters to show all available logs, or add rules to apply a new date/time range:

Filters

Use filters to refine the list based on the value associated with a particular column. Currently, you can filter logs by:

• Started at date

• Trigger (schedule, manual, API, etc.)

• Has warnings

• Status

To define a filter, click the filter icon associated with a column - for example:

...then make selections from the options provided - for example:

Show stats

The show stats option can be toggled ON to view additional columns in the log table:

When this option is toggled ON, three more columns are shown - these show insights for the performance of the process flow.

Run status

The status of a run will be one of the following:

Warnings

If warnings are given during a run (be it failed or successful), the entry is shown with an icon in the Has warnings column - for example:

If the run failed, you should view associated logs to check these warnings, then make any changes necessary to your process flow. If the run succeeded, you should still view the logs for warnings in case the issue causes a failure in the future.

Checking the reason for a failed run

Run logs can contain a lot of information and there may be times when you want to quickly check why a run has failed.

When a process flow run is initialised but fails somewhere in the run, its status is set to failed and an expandable arrow is shown to the left of the entry - click this arrow to view a summary of why the failure occurred. For example:

Additional options

Each log entry has an ellipses icon, with access to additional options:

From here you can:

• View logs. Access more insights and detailed logs for this run. For more information please see our viewing logs page.

• View logs (classic). If you prefer to work with 'original style' logs, you can access them from here. For more information please see our viewing logs (classic) page.

• Download logs. Use this option to download detailed logs in CSV format. For more information please see our downloading run logs page.

• Retry the flow run. Use this option to retry a process flow run that previously failed. For more information please see our retrying a failed process flow run page.

Related information

• Viewing logs

• Viewing logs (classic)

• Downloading run logs

• Retrying a failed process flow run

• Working with your run queue

• Email notifications for failed process flow runs

Introduction

If a process flow run fails for any reason, it's displayed in run logs with a status of failed - for example:

You can retry the run from this page. Before doing so, please consider:

• It's advisable to check your process flow setup (filters and de-dupe shapes) to ensure that running the flow again will not duplicate any runs that may have succeeded in the interim period.

• If a process flow includes references to variables, be aware that any values for those variables may not be the same as they would have been if the failed flow had been successful.

• When the retry option is used, the flow version that was set originally is run. If you want to run an updated version of the process flow, you can choose to edit the flow and initialise a run manually instead.

• A process flow must be enabled for the retry option to work.

The steps

Step 1 Log into the Patchworks dashboard and access your run logs & queue.

Step 2 Click the ellipses icon to access additional options for the log entry, then select the retry flow run option - for example:

About retry timings

When the retry option is used for a failed process flow run, the flow run is dispatched immediately*, using the current time to calculate any relative date filters and settings defined in the flow - unlike scheduled process flow runs which are added to a queue.

To ensure that any time elapsed between the time a failed run was originally queued and the time it is retried, the current time for a retry always assumes the queued at time for the original run that failed.

Example

Suppose that a process flow is scheduled (via a trigger shape) to start at 13:00.

• At 13:00, the run is initialised and added to a queue. The queued at time is set to 13:00.

• At 13:03 the queued run is processed. Because the use queued time process flow setting is toggled ON, the queue delay of 3 minutes is accounted for in all relative date/time settings defined in the flow.

• The run fails.

• At 17:00, the run is retried (via the run logs page). The retry happens immediately however, the current time assumes the original queued at time of 13:00.

Introduction

If required, you can download run logs for a completed process flow run in CSV format, for use outside of Patchworks.

Need to know

Run log exports are completed in CSV format. The exported file includes two columns with level and message headers. For example:

The steps

To download run logs for a specific process flow run, follow the steps below.

Step 2 Click the ellipses icon to access additional options for the log entry, then select the download logs option - for example:

Step 3 The download job is added to a queue and a confirmation message is displayed:

Step 4 When your download is ready, you'll receive an email which includes a link to retrieve the file from the file downloads page. If you can't/don't want to use this link, you can access this page manually by selecting the settings option:

...followed by the file downloads option:

Step 5 On the file downloads page, you'll find any exports that have been completed for your company profile in the last hour. Click the download button for your job:

Step 6 The associated CSV file is saved to the default downloads folder for your browser.