> For the complete documentation index, see [llms.txt](https://doc.wearepatchworks.com/product-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.wearepatchworks.com/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/understanding-how-data-flows-through-shapes.md).

# Understanding how data flows through shapes

## Introduction

Patchworks process flows are incredibly flexible. With a range of shapes for receiving, paginating, manipulating, batching, splitting, caching and sending data, you can build highly complex flows in a matter of a minutes.

With this in mind, it's important to understand how data flows through shapes.  

## How data flows

In the simplest of scenarios, a process flow receives a single payload of unpaginated data and this flows all the way through to completion with no manipulation or batching - one payload is received, processed, and completed.

However, if your incoming data is paginated and/or you introduce shapes capable of generating multiple payloads, it's important to understand how these pass through the flow. Essentially, any payloads that a shape outputs are added to a 'bucket' and it's that bucket that is then passed to the next shape. 

So, all payloads from one shape are passed to the next shape in the same context - they don't pass down the entire flow individually. 

{% hint style="info" %}
If a 'pull' [connection shape](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/connector-shape.md) is configured to use an endpoint that paginates data, the connection shape outputs each page in its own payload.
{% endhint %}

## Demo

The animation below shows how this works.

{% embed url="<https://youtu.be/-kEHeihSSfY>" %}

<details>

<summary><img src="https://2440044887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLYNcUBVQwSkOMG6KjZfz%2Fuploads%2FfUVdhc1UgMWObOesLjkS%2Ficon%20eye.png?alt=media&amp;token=c8b6d3a9-b5a9-4cb0-8aeb-00bd39c00b60" alt="" data-size="line"> Suggested view options</summary>

For best results, view this animation in full screen and adjust settings to 2160p 4k:

<img src="https://2440044887-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FLYNcUBVQwSkOMG6KjZfz%2Fuploads%2F4pf31JsnR855ISKiGDVQ%2Fyoutube%20settings.png?alt=media&amp;token=fb0d187b-d94a-4045-929f-6fdc8dc850c3" alt="" data-size="original">

</details>

## Shape timeouts and retries

### Timeouts

All shapes (except the [connection shape](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/connector-shape.md)) have a set timeout of 30 minutes. If processing is not completed within this time, the shape fails.&#x20;

The timeout for a connection shape is configurable via [connection shape settings](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/connector-shape.md#configuring-settings-for-a-connection-shape).

### Retries

With some exceptions (detailed below), a further three attempts will be made if a process flow shape fails. Exceptions are summarised below:

| Shape                                                                                                                                                                      | Number of retries |
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- |
| [Add to cache](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/cache/add-to-cache-shape.md) | 1                 |
| [Connection](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/connector-shape.md)            | 2                 |
| [Manual payload](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/manual-payload-shape.md)   | 1                 |
| [Split](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/standard-shapes/split-shape.md)                     | 1                 |
| [De-dupe](/product-documentation/~/changes/J8IbZkP6ASUZu2oBhGi2/process-flows/building-process-flows/process-flow-shapes/advanced-shapes/de-dupe-shape.md)                 | 1                 |