# Generating dynamic cache keys with variables ## Introduction When an [add to cache shape](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/cache/add-to-cache-shape) is dropped into a process flow, the entire incoming payload is cached and associated with the given **cache key**. Depending on the cache type, you can load this cache later in the same flow or in a different flow. In the simplest scenario, your given cache key would be a static value (e.g. `customers`) and you would use this to load the entire cache (containing perhaps tens, hundreds, even thousands of items) where required. But what if you want to load a specific item from a cache, rather than the whole thing? This is where **dynamic cache keys** are so useful. ## How it works To load data from a cache, you configure a **load from cache** shape with the required `cache` and a single `cache key`. All data associated with your given `cache key` is loaded. Consider the example incoming payload below, where four records are cached with a static `cache key` with a value of `customers`: {% code title="cache key: customers" lineNumbers="true" %} ```json [ { "id": 1000000001, "first_name": "Jane", "last_name": "Smith", "items": { "itemref": "0000001", "item1": "apples", "item2": "oranges", "item3": "pears" } }, { "id": 1000000002, "first_name": "George", "last_name": "Jones", "items": { "itemref": "0000002", "item1": "tangerines", "item2": "peaches", "item3": "grapes" } }, { "id": 1000000003, "first_name": "Bob", "last_name": "Brown", "items": { "itemref": "0000003", "item1": "nectarines", "item2": "raspberries", "item3": "strawberries" } }, { "id": 1000000004, "first_name": "Marjorie", "last_name": "Simpson", "items": { "itemref": "0000004", "item1": "blueberries", "item2": "cranberries", "item3": "apricots" } } ] ``` {% endcode %} If we were to configure a **load from cache** shape to access the `customers` cache key, all four records would be loaded. So, in order to load specific items from a cache, the incoming data must be added to a cache in such a way that we can easily target individual items. We need an efficient way to take incoming data, batch it into single-record payloads and add each of these to the cache with its own unique, identifying **cache key** - i.e.:

cache: customerData
cache key: customer-1000000001

[
    {
        "id": 1000000001,
        "first_name": "Jane",
        "last_name": "Smith",
        "items": {
           "itemref": "0000001",
            "item1": "apples",
            "item2": "oranges",
            "item3": "pears"
        }
    }
]

cache: customerData
cache key: customer-1000000002

[
    {
        "id": 1000000002,
        "first_name": "George",
        "last_name": "Jones",
        "items": {
           "itemref": "0000002",
            "item1": "tangerines",
            "item2": "peaches",
            "item3": "grapes"
        }
    }
]

cache: customerData
cache key: customer-1000000003

[
    {
        "id": 1000000003,
        "first_name": "Bob",
        "last_name": "Brown",
        "items": {
           "itemref": "0000003",
            "item1": "nectarines",
            "item2": "raspberries",
            "item3": "strawberries"
        }
    }
]

cache: customerData
cache key: customer-1000000004

[
    {
        "id": 1000000004,
        "first_name": "Marjorie",
        "last_name": "Simpson",
        "items": {
           "itemref": "0000004",
            "item1": "blueberries",
            "item2": "cranberries",
            "item3": "apricots"
        }
    }
]
We can achieve this as follows:
ActionOutcome#OutcomeOutcomeOutcomeStep
Place a flow control shape immediately before the add to cache shape and configure it to create batches of 1 at the appropriate level for your data.The incoming payload is batched into multiple payloads - one payload per data element (e.g. one order per payload, one customer per payload, one product per payload, etc.).1The subsequent add to cache shape receives multiple payloads, each containing a single record.Incoming data is batched into single-record payloads and these are sent on to the subsequent add to cache shape.The incoming payload is batched into multiple payloads - one payload per data element (e.g. one order per payload, one customer per payload, one product per payload, etc.). 1
Configure the add to cache shape and specify a payload variable as the cache key, where the variable looks for the first occurrence of a uniquely identifying element in the payload (typically an id or reference number).The add to cache shape receives and caches single-record payloads from the flow control shape. The cache key for each payload is generated dynamically by resolving the payload variable from each incoming payload.2Each time a payload hits the add to cache shape, it gets added to the cache with its own cache key, resolved from the given payload variable. Now we can easily target single or multiple records to be loaded via a load from cache shape.Multiple payloads are received from the flow control shape. Each one is added to the cache with its own cache key, resolved from the given payload variable. These cache keys can now be referenced by subsequent load from cache shapes. Since each cache key contains a single record, we now have a way to target specific records from our original payload.The add to cache shape receives and caches single-record payloads from the flow control shape - the cache key for each payload is generated dynamically by resolving the payload variable for each incoming payload. 2
{% hint style="info" %} [Flow control](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/standard-shapes/flow-control-shape) is an easy way to batch incoming data into single-record payloads, however you may prefer an alternative approach. The important point is that the add to cache shape must receive single-record payloads - how you achieve this is up to you. {% endhint %} ## Need to know * When you specify a dynamic variable as the **cache key**, the value for that variable is injected into the key. To prevent the case where large amounts of data are passed into the key, there is a character limit is 128 characters. * Any combination of [payload](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/dynamic-variables/payload-variables), [flow](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/dynamic-variables/flow-variables) and [metadata](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/dynamic-variables/metadata-variables) variables can be used to form cache key names. * When using dynamic variables for auto-generated cache keys, resolved variable values must not contain full stop (.) or colon (:) characters. ## Using a variable to generate dynamic cache keys Follow the steps below to configure an **add to cache** shape with a [payload variable](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/dynamic-variables/payload-variables) for generating dynamic cache keys. {% hint style="info" %} These steps assume that you have already defined a **flow control** shape (or some other means) to ensure that the **add to cache** shape receives single-record payloads. {% endhint %} **Step 1**\ [Drop an add to cache shape into your process flow](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/cache/add-to-cache-shape/..#adding-an-add-to-cache-shape-to-a-process-flow), where required.
Example In the example below, we have an incoming payload which includes orders in an `orders` array: ![](https://2440044887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLYNcUBVQwSkOMG6KjZfz%2Fuploads%2Fdkp8nCNrIeJBDviHOiVf%2Fpayload%201.png?alt=media\&token=81022e8c-4710-4e27-b1de-6271250db0ba) If we want to add these orders to a cache, we need to split them into batches of 1, so our **flow control** shape is configured as below: ![](https://2440044887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLYNcUBVQwSkOMG6KjZfz%2Fuploads%2FLJozBKJs2zaajMb0cr97%2Fpayload%202.png?alt=media\&token=9c2669eb-945b-45b4-8fc3-d6e7b6e916d0) This will generate two payloads. So, if we go on to drop an **add to cache** shape immediately after, each of these payloads will be cached (and therefore retrieved) separately.
**Step 2**\ In the **add to cache** shape settings, choose to **create cache**:
**Step 3**\ Set the **cache level** and **name** as required and save changes.
{% hint style="info" %} For more information on these fields please see the [add to cache shape](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/cache/add-to-cache-shape/..#adding-an-add-to-cache-shape-to-a-process-flow) page. {% endhint %} **Step 4**\ Select the cache that you just created - for example:
**Step 5**\ Move down to the **cache key** field and enter the required key. Here, you use standard [payload variable syntax ](https://doc.wearepatchworks.com/product-documentation/process-flows/dynamic-variables/payload-variables#payload-variable-syntax)to define your target data element: {% code lineNumbers="true" %} ``` [[payload.]] ``` {% endcode %} ...where `schema notation` should be replaced with the notation path to the first occurrence of the required element in the payload which should be used to form the cache key. If required, you can also include a static prefix or suffix. For example: {% code lineNumbers="true" %} ``` customer-[[payload.0.id>]] ``` {% endcode %} The output of the payload variable will be used as the **cache key**.
Example Consider the payload below: {% code lineNumbers="true" %} ```json { "orders": [ { "id": 1, "customer": { "email": "joe.bloggs@wearepatchworks.com" } } ] } ``` {% endcode %} ...and **our cache key** is specified as: {% code lineNumbers="true" %} ``` customer-[[payload.0.id>]] ``` {% endcode %} The generated **cache key** would be: {% code lineNumbers="true" %} ``` customer-1 ``` {% endcode %} ...where `1` is the id associated with the first object in the `orders` array.The
{% hint style="info" %} Our example uses dynamic payload variables however, you can also use metadata variables and/or flow variables. For more information please see [dynamics variables](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/dynamic-variables) section. {% endhint %} {% hint style="warning" %} When using dynamic variables for auto-generated cache keys, ensure that resolved variable values do not contain full stop (.) or colon (:) characters. For example: `[[payload.id]]` resolving to `1234567` is good\ `[[payload.id]]` resolving to `123.4567` is bad {% endhint %} **Step 6**\ Save the **add to cache** shape settings. ## Loading data from a cache Cached data can be loaded via our load from cache shape. Please refer to the [Load from cache shape](https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/cache/load-from-cache-shape) section for more information.