# Flow control shape

## Introduction

The **flow control** shape can be used for cases where you're pulling lots of records from a source connection, but your target connection needs to receive data in small batches. Two common use cases for this shape are:

* A target system can only accept items one at a time
* A target system has a maximum number of records that can be added/updated at one time

The **flow control** shape takes all received items, splits them into batches of your given number, and sends these batches into the target connection.   

{% hint style="info" %}
A single incoming payload for [any process flow shape](https://open.gitbook.com/~site/site_dIV1g/~/revisions/9dTvdvRJIRVZQXuBMMnJ/process-flows/building-process-flows/process-flow-shapes) should not exceed 500MB.

We recommend processing multiple, smaller payloads rather than one single payload (1000 x 0.5MB payloads are more efficient than 1 x 500MB payload!).

For payloads up to 500MB, consider adding a[ flow control shape](https://open.gitbook.com/~site/site_dIV1g/~/revisions/9dTvdvRJIRVZQXuBMMnJ/process-flows/building-process-flows/process-flow-shapes/standard-shapes/flow-control-shape) to batch data into multiple, smaller payloads. Payloads exceeding 500MB should be batched at source.
{% endhint %}

## Demo

{% embed url="<https://www.youtube.com/watch?v=NiVxv6lV1zU>" %}

## Adding & configuring a flow control shape

**Step 1**\
In your process flow, add the **flow control** shape in the usual way:

<div align="left"><figure><img src="/files/3FhLoWEoY3eBUHxq053L" alt="" width="375"><figcaption></figcaption></figure></div>

**Step 2**\
Select a **source integration** and **endpoint** to determine where the incoming payload to be split originates - for example:

<div align="left"><figure><img src="/files/kSb1Kp9qoHLhMefFZV71" alt="" width="355"><figcaption></figcaption></figure></div>

**Step 3**\
Move down to the **batch level** field and select that data element that you are putting into batches. For example :

<div align="left"><figure><img src="/files/flxW6lCUAX0OEi7YVMEb" alt="" width="357"><figcaption></figcaption></figure></div>

{% hint style="info" %}
The data structure in this dropdown field is pulled from schema associated with the source. If your data is received from a non-connector source (e.g. manual payload, API, webhook, etc.) then you can toggle ON the **manual input** option and enter the data path manually).
{% endhint %}

**Step 4**\
In the **batch size** field, enter the number of items to be included in each batch. For example:

<div align="left"><figure><img src="/files/l6Tq2WgykX0ZzhNOQ131" alt="" width="356"><figcaption></figcaption></figure></div>

**Step 5**\
By default, the **payload format** is **auto**-detected but you can set a specific format here if you prefer:

<div align="left"><figure><img src="/files/jetiW3xkvSTsVagfT7pb" alt="" width="352"><figcaption></figcaption></figure></div>

**Step 6**

If you're creating batches of one record, you can toggle ON the  `Do not wrap single records in an array` option if you want the output to be this:

```json
{
    "field": "value",
    "field": "value",
    "field": "value"
}
```

...rather than this:

```json
[{
    "field": "value",
    "field": "value",
    "field": "value"
}]
```

**Step 7**\
Save the shape. Now when you run this process flow, data will be split into batches of your given size.&#x20;

{% hint style="info" %}
If you check the payload for the flow control step after it has run, you'll see that there's one payload for every batch created. For example:

<img src="/files/Bpj9LQdj7RxGO9cvYeLj" alt="" data-size="original">
{% endhint %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://doc.wearepatchworks.com/product-documentation/process-flows/building-process-flows/process-flow-shapes/standard-shapes/flow-control-shape.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.