Generating dynamic cache keys with variables

Introduction

When an add to cache shape is dropped into a process flow, the entire incoming payload is cached and associated with the given cache key. Depending on the cache type, you can load this cache later in the same flow or in a different flow.

In the simplest scenario, your given cache key would be a static value (e.g. customers) and you would use this to load the entire cache (containing perhaps tens, hundreds, even thousands of items) where required. But what if you want to load a specific item from a cache, rather than the whole thing?

This is where dynamic cache keys are so useful.

How it works

To load data from a cache, you configure a load from cache shape with the required cache and a single cache key. All data associated with your given cache key is loaded.

Consider the example incoming payload below, where four records are cached with a static cache key with a value of customers:

cache key: customers
[
    {
        "id": 1000000001,
        "first_name": "Jane",
        "last_name": "Smith",
        "items": {
           "itemref": "0000001",
            "item1": "apples",
            "item2": "oranges",
            "item3": "pears"
        }
    },
    {
        "id": 1000000002,
        "first_name": "George",
        "last_name": "Jones",
        "items": {
           "itemref": "0000002",
            "item1": "tangerines",
            "item2": "peaches",
            "item3": "grapes"
        }
    },
    {
        "id": 1000000003,
        "first_name": "Bob",
        "last_name": "Brown",
        "items": {
           "itemref": "0000003",
            "item1": "nectarines",
            "item2": "raspberries",
            "item3": "strawberries"
        }
    },
    {
        "id": 1000000004,
        "first_name": "Marjorie",
        "last_name": "Simpson",
        "items": {
           "itemref": "0000004",
            "item1": "blueberries",
            "item2": "cranberries",
            "item3": "apricots"
        }
    }
]

If we were to configure a load from cache shape to access the customers cache key, all four records would be loaded.

So, in order to load specific items from a cache, the incoming data must be added to a cache in such a way that we can easily target individual items. We need an efficient way to take incoming data, batch it into single-record payloads and add each of these to the cache with its own unique, identifying cache key - i.e.:

We can achieve this as follows:

Action
Outcome

The incoming payload is batched into multiple payloads - one payload per data

element (e.g. one order per payload, one

customer per payload, one product per payload, etc.).

Configure the add to cache shape and specify a payload variable as the cache key, where the variable looks for the first occurrence of a uniquely identifying element in the payload (typically an id or reference number).

The add to cache shape receives and

caches single-record payloads from the flow

control shape. The cache key for each

payload is generated dynamically by

resolving the payload variable from each

incoming payload.

Flow control is an easy way to batch incoming data into single-record payloads, however you may prefer an alternative approach. The important point is that the add to cache shape must receive single-record payloads - how you achieve this is up to you.

Need to know

  • When you specify a dynamic variable as the cache key, the value for that variable is injected into the key. To prevent the case where large amounts of data are passed into the key, there is a character limit is 128 characters.

  • Any combination of payload, flow and metadata variables can be used to form cache key names.

Using a variable to generate dynamic cache keys

Follow the steps below to configure an add to cache shape with a payload variable for generating dynamic cache keys.

These steps assume that you have already defined a flow control shape (or some other means) to ensure that the add to cache shape receives single-record payloads.

Step 1 Drop an add to cache shape into your process flow, where required.

Step 2 In the add to cache shape settings, choose to create cache:

Step 3 Set the cache level and name as required and save changes.

For more information on these fields please see the add to cache shape page.

Step 4 Select the cache that you just created - for example:

Step 5 Move down to the cache key field and enter the required key. Here, you use standard payload variable syntax to define your target data element:

[[payload.<schema notation>]]

...where schema notation should be replaced with the notation path to the first occurrence of the required element in the payload which should be used to form the cache key. If required, you can also include a static prefix or suffix. For example:

customer-[[payload.0.id>]]

The output of the payload variable will be used as the cache key.

Our example uses dynamic payload variables however, you can also use metadata variables and/or flow variables. For more information please see dynamics variables section.

Step 6 Save the add to cache shape settings.

Loading data from a cache

Cached data can be loaded via our load from cache shape. Please refer to the Load from cache shape section for more information.

PreviousAdd to cache shapeNextAppending data to a cache

Last updated