Custom scripting technical overview

Introduction

This page provides some technical background for those interested in how the Patchworks custom scripting feature is implemented.

Need to know

• The maximum memory size for a custom script is 512MB

• The maximum size of a custom script is 4GB

Patchworks & OpenFaas overview

Patchworks custom scripting code is packaged into containers and run as serverless functions.

This is achieved using OpenFaas (Function as a Service) - a framework for building serverless functions on top of containers (with Docker and Kubernetes).

By utilising Docker containers, OpenFaaS enables language-agnostic execution of code. By wrapping code in a container, a self-contained environment is created - this environment includes your code, its dependencies, and any necessary runtime configurations.

The container acts as an isolated unit, providing consistent and predictable execution irrespective of the underlying infrastructure. The containerisation process involves creating a Docker image which can be easily deployed and executed on any system that supports Docker - thereby ensuring portability and compatibility across different environments.

This approach eliminates concerns about specific runtime dependencies and configurations on the host system and as such, no restrictions on the scope of programming languages that can be used are imposed. It's this flexibility that allows Patchworks to provide the functionality for customers to choose the most suitable language for their individual use cases.

The capability to support multiple languages is hugely advantageous. For example, if you have a workflow that involves processing data in Python, performing image manipulation in Node.js, and generating reports in Go, you can create separate functions for each task using the required languages. OpenFaaS abstracts away the underlying infrastructure details, allowing these functions to seamlessly interact with each other and the Patchworks iPaaS platform.

Script creation & execution

The diagram below provides a simplified illustration of what happens when a custom script is created and then executed.