2024 07 24 release notes (core)
Release summary
This page summarises items scheduled for the following release to the Patchworks Production environment:
Patchworks core (backend)
1.22.0
24th July @ 9am
Patchworks dashboard (frontend)
1.16.0
24th July @ 9am
This release includes possible breaking changes related to the validation of process flow filters and transforms. Please ensure that you check the Breaking changes section.
New features, updates, and enhancements of note are summarised by product area, below:
Process flows
Operations Historically, Patchworks usage has been tied to the number of transactions processed. In this release, transaction counts are retired and the concept of operations is introduced.
Operations are a simpler, more visible way to track your use of the Patchworks platform. An operation is counted whenever a request is made (successfully or otherwise) to send or receive a payload to/from an endpoint. This may be via connector shapes, webhooks, events, or using the Patchworks API.
You can view aggregated operations for a month via your Company insights page and per process flow run via Run logs.
CPT-3062
Process flow queues How process flow run jobs are queued has been improved to help ensure that race-condition scenarios are avoided.
Previously, it was possible to hit a race condition with flows, where an ever-increasing number of flows would simultaneously drain and compete for resources with other running flows that use the same connector instances. In this release, a concurrent process flow run limit is enforced per company - so only a certain number of flows (as determined by your subscription tier) can run simultaneously.
CPT-3157
Field mapping string transforms With two exceptions (below), all string-type transform functions will now allow empty strings to be passed in. Previously, if a string transform received an empty value, the map step would fail.
The two exceptions are country name and country code. These transform functions will still cause the map step to fail if empty values are received.
CPT-3192
Filters: enforced validation for data types Additional validation has been added for filters (wherever applied), meaning that input values must be of the expected data type.
This is a phased change. Phase 1 was implemented on the 17th July 2024. Phase 2 is implemented in this release. For more information please see the Breaking changes section.
CPT-2862
Transforms: enforced validation for data types Additional validation has been added to all transform functions, which means that input values must be of the expected type. For example, if a transform function is expecting a number value for its input and a string is provided, validation will fail. This is a phased change. Phase 1 was implemented on the 21st March 2024. Phase 2 is implemented in this release. For more information please see the Breaking changes section.
CPT-2982
Connector instance timezone & optional filters Fixed an issue where the timezone selected for a connector instance was not reflected when optional relative date filters were applied in the process flow connector shape.
CPT-3144
Initialise flow with data & a relative time variable Fixed an issue where a process flow variable defined with a relative time variable did not resolve as expected when the process flow was run via the Initialise flow with data option.
CPT-3155
Breaking changes
This release includes changes that may cause process flow runs to produce errors, or to fail. Please follow the advice below to avoid any issues:
(1) Filter validation enforcement
When defining process flow filters (for example, in a filter or route shape), you select a data field and then the corresponding data type for this field - for example:
In this release, we start to enforce the specified data type for any filters. For example, if a filter based on the customer_id field is defined to expect string data, but the incoming payload provides these values as numbers, this will cause a validation error.
What's happening?
Filter validation issues will be enforced in two phases:
2: Enforcement
If a process flow runs and an invalid data type is passed into a filter, an error message will be displayed in the run logs and the process flow run will fail.
24th July 2024
Phase 1 (recommended actions prior to this release)
Phase 1 was implemented on the 17th July 2024. We recommended making the following checks before this time:
Filter & route shapes
Check any filter and route shapes in your process flows now, and ensure that defined data types reflect your source data. If you find a mismatch, you can:
Change the data type for the process filter, if appropriate
Apply a field transform (via a map shape) immediately before the filter step to convert incoming values to a different data type. Several transform functions are available which can help to change data from one type to another - please see Working with field mapping transformations for more information.
Change the data type for this value at source (if you have the option to do so)
Connector shapes
Check any connector shapes in your process flows, and ensure that defined data types reflect your source data. If you find a mismatch, action depends on whether the connector is a standard Patchworks connector or a custom connector that was built by your own company with the Connector builder:
For a standard Patchworks connector, it’s advisable to change the data type for this value at source. If this isn’t possible, you can edit the filter within the connector endpoint setup and change the data type to match source data. However, be aware that if you subsequently install a Patchworks update for this connector, your changes will be overwritten.
For custom connectors, you can edit the filter within the connector endpoint setup and change the data type to match source data.
Phase 1 (recommended actions ASAP after this release)
After the release on the 17th July 2024, please review run logs for all of your active process flows and check if there are any warnings - for example:
You can filter your run logs list so that only entries with warnings are displayed.
If warnings are given, please check the associated process flow and make changes as detailed above.
Phase 2 action required
Phase 2 is implemented in this release. We recommend checking run logs for your flows with extra care as soon as possible after this release, just in case validation issues are still present.
(2) Transform validation enforcement
Similar to the filter validation enforcement change detailed above, this release enforces the specified data type for field mapping transform functions.
For example, if a string > lowercase transform is defined to convert the customer_id field to lowercase but the incoming payload provides these values as numbers, this will cause a validation error.
What's happening?
Transform validation issues will be enforced in two phases:
1: Warning
If a process flow runs and an invalid data type is passed into a transform function, a warning message is displayed in the run logs. This message includes the name of the transform function that needs to be checked/updated. For example:
Any issues identified in this way will not cause the process flow to fail at this time. However, if identified issues are not resolved, the process flow will fail when Phase 2 of this change is released.
Note: This phase was implemented on the 21st March 2024.
2: Enforcement
24th July 2024
Action required
With the implementation of Phase 1 on the 21st March 2024, you will have been receiving warning messages in run logs in the event of any transform validation errors.
We recommend checking run logs for your flows with extra care before and after this release, just in case validation issues are still present.
If warnings are present, check any map shapes in relevant process flows and look for transform functions. Where found, check whether an expected data type is specified and if yes, ensure that defined data types reflect your source data. If you find a mismatch, you can:
Change the data type associated with this value at source (if you have the option to do so).
Add a transform function immediately before, changing source data to the required type. For example, cast a string to a number or a number to a string.
Use a different transform function (if applicable).
Last updated
