2024 07 24 release notes (core)

Release summary

This page summarises items scheduled for the following release to the Patchworks Production environment:

Patchworks core (backend)

Release version	Release date
1.22.0	24th July @ 9am

Patchworks dashboard (frontend)

Release version	Release date
1.16.0	24th July @ 9am

This release includes possible breaking changes related to the validation of process flow filters and transforms. Please ensure that you check the Breaking changes section.

New features, updates, and enhancements of note are summarised by product area, below:

• Process flows

• Breaking changes

Process flows

Type	Summary	Ref #
New	Operations Historically, Patchworks usage has been tied to the number of transactions processed. In this release, transaction counts are retired and the concept of operations is introduced. Operations are a simpler, more visible way to track your use of the Patchworks platform. An operation is counted whenever a request is made (successfully or otherwise) to send or receive a payload to/from an endpoint. This may be via connector shapes, webhooks, events, or using the Patchworks API. You can view aggregated operations for a month via your Company insights page and per process flow run via Run logs. About operations	CPT-3062
Enhancement	Process flow queues How process flow run jobs are queued has been improved to help ensure that race-condition scenarios are avoided. Previously, it was possible to hit a race condition with flows, where an ever-increasing number of flows would simultaneously drain and compete for resources with other running flows that use the same connector instances. In this release, a concurrent process flow run limit is enforced per company - so only a certain number of flows (as determined by your subscription tier) can run simultaneously. About concurrent processing	CPT-3157
Change	Field mapping string transforms With two exceptions (below), all string-type transform functions will now allow empty strings to be passed in. Previously, if a string transform received an empty value, the map step would fail. The two exceptions are country name and country code. These transform functions will still cause the map step to fail if empty values are received.	CPT-3192
Breaking change	Filters: enforced validation for data types Additional validation has been added for filters (wherever applied), meaning that input values must be of the expected data type. This is a phased change. Phase 1 was implemented on the 17th July 2024. Phase 2 is implemented in this release. For more information please see the Breaking changes section.	CPT-2862
Breaking change	Transforms: enforced validation for data types Additional validation has been added to all transform functions, which means that input values must be of the expected type. For example, if a transform function is expecting a number value for its input and a string is provided, validation will fail. This is a phased change. Phase 1 was implemented on the 21st March 2024. Phase 2 is implemented in this release. For more information please see the Breaking changes section. Transform functions	CPT-2982
Bug	Connector instance timezone & optional filters Fixed an issue where the timezone selected for a connector instance was not reflected when optional relative date filters were applied in the process flow connector shape. Connector shape	CPT-3144
Bug	Initialise flow with data & a relative time variable Fixed an issue where a process flow variable defined with a relative time variable did not resolve as expected when the process flow was run via the Initialise flow with data option. The process flow canvas	CPT-3155

Breaking changes

This release includes changes that may cause process flow runs to produce errors, or to fail. Please follow the advice below to avoid any issues:

• (1) Filter validation enforcement

• (2) Transform validation enforcement

(1) Filter validation enforcement

When defining process flow filters (for example, in a filter or route shape), you select a data field and then the corresponding data type for this field - for example:

In this release, we start to enforce the specified data type for any filters. For example, if a filter based on the customer_id field is defined to expect string data, but the incoming payload provides these values as numbers, this will cause a validation error.

What's happening?

Filter validation issues will be enforced in two phases:

Phase	What will happen	When
1: Warning	If a process flow runs and an invalid data type is passed into a filter, the run logs will display an error message. This message will include the ‘problem’ data type - for example: The process flow will continue to run however, it’s possible that your filters will not be applied as expected. If identified issues are not resolved, the process flow will fail when Phase 2 of this change is released.	17th July 2024
2: Enforcement	If a process flow runs and an invalid data type is passed into a filter, an error message will be displayed in the run logs and the process flow run will fail.	24th July 2024

Phase 1 (recommended actions prior to this release)

Phase 1 was implemented on the 17th July 2024. We recommended making the following checks before this time:

Filter & route shapes

Check any filter and route shapes in your process flows now, and ensure that defined data types reflect your source data. If you find a mismatch, you can:

• Change the data type for the process filter, if appropriate

• Apply a field transform (via a map shape) immediately before the filter step to convert incoming values to a different data type. Several transform functions are available which can help to change data from one type to another - please see Working with field mapping transformations for more information.

• Change the data type for this value at source (if you have the option to do so)

Connector shapes

Check any connector shapes in your process flows, and ensure that defined data types reflect your source data. If you find a mismatch, action depends on whether the connector is a standard Patchworks connector or a custom connector that was built by your own company with the Connector builder:

• For a standard Patchworks connector, it’s advisable to change the data type for this value at source. If this isn’t possible, you can edit the filter within the connector endpoint setup and change the data type to match source data. However, be aware that if you subsequently install a Patchworks update for this connector, your changes will be overwritten.

• For custom connectors, you can edit the filter within the connector endpoint setup and change the data type to match source data.

Phase 1 (recommended actions ASAP after this release)

After the release on the 17th July 2024, please review run logs for all of your active process flows and check if there are any warnings - for example:

You can filter your run logs list so that only entries with warnings are displayed.

If warnings are given, please check the associated process flow and make changes as detailed above.

Phase 2 action required

Phase 2 is implemented in this release. We recommend checking run logs for your flows with extra care as soon as possible after this release, just in case validation issues are still present.

(2) Transform validation enforcement

Similar to the filter validation enforcement change detailed above, this release enforces the specified data type for field mapping transform functions.

For example, if a string > lowercase transform is defined to convert the customer_id field to lowercase but the incoming payload provides these values as numbers, this will cause a validation error.

What's happening?

Transform validation issues will be enforced in two phases:

Phase	What will happen	When
1: Warning	If a process flow runs and an invalid data type is passed into a transform function, a warning message is displayed in the run logs. This message includes the name of the transform function that needs to be checked/updated. For example: Any issues identified in this way will not cause the process flow to fail at this time. However, if identified issues are not resolved, the process flow will fail when Phase 2 of this change is released. Note: This phase was implemented on the 21st March 2024.	21st March 2024
2: Enforcement	If a process flow runs and an invalid data type is passed into a filter, an error message will be displayed in the run logs and the process flow run will fail - for example:	24th July 2024

Action required

With the implementation of Phase 1 on the 21st March 2024, you will have been receiving warning messages in run logs in the event of any transform validation errors.

We recommend checking run logs for your flows with extra care before and after this release, just in case validation issues are still present.

If warnings are present, check any map shapes in relevant process flows and look for transform functions. Where found, check whether an expected data type is specified and if yes, ensure that defined data types reflect your source data. If you find a mismatch, you can:

• Change the data type associated with this value at source (if you have the option to do so).

• Add a transform function immediately before, changing source data to the required type. For example, cast a string to a number or a number to a string.

• Use a different transform function (if applicable).