Introduction

This page provides some technical background for those interested in how the Patchworks custom scripting feature is implemented.

Need to know

• The maximum memory size for a custom script is 512MB

• The maximum size of a custom script is 4GB

Patchworks & OpenFaas overview

Patchworks custom scripting code is packaged into containers and run as serverless functions.

This is achieved using OpenFaas (Function as a Service) - a framework for building serverless functions on top of containers (with Docker and Kubernetes).

By utilising Docker containers, OpenFaaS enables language-agnostic execution of code. By wrapping code in a container, a self-contained environment is created - this environment includes your code, its dependencies, and any necessary runtime configurations.

The container acts as an isolated unit, providing consistent and predictable execution irrespective of the underlying infrastructure. The containerisation process involves creating a Docker image which can be easily deployed and executed on any system that supports Docker - thereby ensuring portability and compatibility across different environments.

This approach eliminates concerns about specific runtime dependencies and configurations on the host system and as such, no restrictions on the scope of programming languages that can be used are imposed. It's this flexibility that allows Patchworks to provide the functionality for customers to choose the most suitable language for their individual use cases.

The capability to support multiple languages is hugely advantageous. For example, if you have a workflow that involves processing data in Python, performing image manipulation in Node.js, and generating reports in Go, you can create separate functions for each task using the required languages. OpenFaaS abstracts away the underlying infrastructure details, allowing these functions to seamlessly interact with each other and the Patchworks iPaaS platform.

Script creation & execution

The diagram below provides a simplified illustration of what happens when a custom script is created and then executed.

Introduction

Having selected the scripts option from the left-hand navigation menu, the scripts page is displayed:

This page is split into two panels:

• Installed. Lists all scripts that you have added, or installed from the marketplace.

• Marketplace. Lists all Patchworks scripts that you have not yet installed.

Installing a custom script

Patchworks provides a range of scripts which you can install and use as-is, or take as a starting point for developing your own script. These are available from the lower script marketplace panel. To install a script, follow the steps below.

Step 1 Click the install button associated with the script that you want to use - for example:

Step 2 The script installs and is added to the upper installed section:

As part of the installation procedure, the script is deployed. This means that it's ready for use straight away.

You can now use the script as-is, or make your own changes and deploy the script again once it's ready.

Introduction

This page explains how to add a custom script for payload transformations. You will create a new script, add your code using IntelliSense, and then save changes ready for testing. Adding a new custom script can be broken down into three distinct stages:

1. Create a new script

2. Add script code

3. Test script code

This page walks through creating and testing a new script. We'll be adding a Javascript (node 18) script to change email addresses in a given payload to [email protected]. If you'd like to try this yourself, our script and test payload are available below.

module.exports = async function (data) {
    const payload = JSON. parse (data.payload);

   if (payload.customers) {
     payload.customers.forEach(customer => {
         customer.email = '[email protected]';
      })
    }

    return { "payload": JSON.stringify(payload) };
};

[
    {
        "id": 1000000001,
        "first_name": "Britney",
        "last_name": "Spears",
        "email": "[email protected]"
    },
    {
        "id": 1000000002,
        "first_name": "Madonna",
        "last_name": "Ciccone",
        "email": "[email protected]"
    },
    {
        "id": 1000000003,
        "first_name": "Elvis",
        "last_name": "Presley",
        "email": "[email protected]"
    }
]

Need to know

• The maximum memory size for a custom script is 512MB

• The maximum size of a custom script is 4GB

Creating a new script

Follow the steps below:

Step 1 Log into the Patchworks dashboard and select scripts from the left-hand navigation bar.

Step 2 Select the create script button from the scripts page:

Step 3 Add a unique name and a description for your script, then choose your preferred coding language:

Step 4 Click the create button. The new script is saved and you're now in edit mode, ready to add and test your code:

Adding your code

Follow the steps below:

Step 1 Move down to the code panel.

Step 2 Add your script code (remember that you can use IntelliSense for easy code entry) - for example:

Step 3 When you're ready to test the script, click the save and deploy button:

This saves the code and deploys it to Patchworks, ready for testing/use.

Testing your script

Follow the steps below:

Step 1 Move across to the test panel.

Step 2 Add/paste a payload that can be used to test your script - for example:

Step 3 If your script references variables or flow variables, you can also add variables for testing:

Step 4 Click the run button:

...the script runs against your test payload and results are displayed at the end - for example:

Step 5 Keep testing the script until you're satisfied that it's working as required.

Once these tests are complete, you can:

• Apply a payload-level script to process flows via the script shape

• Apply a field-level custom script to field transformations via the map shape

Introduction

When testing and debugging scripts, it can be useful to write specific information to logs after the script has finished processing.

You can do this by adding a logs array key to a script, with required output added as strings - for example:

...is output as:

Need to know

• Log messages must be strings.

• A maximum of 500 characters (in total) is allowed.

• Use with care (especially in loops) as logs are output every time the script runs.

Introduction

'Out of the box', Patchworks pulls data from a source system and pushes it into a destination system, using field mappings to decide where each item of source data should be placed in the destination system.

Patchworks provides default mappings which can be adjusted if needed, and a comprehensive set of transform functions that can be used if your source data doesn't quite slot neatly into the destination.

However, requirements for manipulating source data can be more complex than any standard set of transform functions can handle. If this is your position and you have developer experience in-house, custom scripting can be used to write and maintain your own scripts.

Using our integrated editor with IntelliSense, you can get as complex as you need with a choice of coding languages.

How are custom scripts used?

Once a script is deployed, it can be selected for use in process flows via:

• (run a script at any point in a process flow)

• (add a to run a script to transform source fields before they are pushed to destination fields)

'Cold storage'

• If a custom script has not been used for 4 days and is NOT associated with an enabled process flow, it is un-deployed and placed in 'cold storage'. If you add an un-deployed script to a process flow (via the or a ) it will be re-deployed automatically when called.

• If a custom script is associated with an enabled process flow, it will never be un-deployed and put into 'cold storage'.

Supported languages

The following coding languages are currently supported for custom scripting:

Supported libraries & packages

Default packages are available for supported coding languages. Additionally:

If you require additional libraries you can:

• embed them in your script directly

• submit a request to Patchworks support for a library to be included globally

Related pages

Introduction

This page includes scripts that may be useful either in and of themselves, or as a start point for your own scripting work.

PHP

C#

This script takes an XML input string, loads it into an XmlDocument, and then converts that XML to a JSON string using the Newtonsoft.Json library.