Connection pools
Last updated
Last updated
Many APIs enforce rate limits for the number of requests that can be made in a given period - typically per minute. Others (e.g. NetSuite) set concurrency limits to determine the number of requests that can be made simultaneously. As with rate limiting, exceeding a concurrency limit can result in delays, throttling, or errors (depending on the system).
In this scenario, we need a way to ensure that requests are only activated when we know that the concurrency limit won't be exceeded. For example, if our concurrency limit is 5 and 5 requests are active, we want to ensure that request number 6 is not made until there's a free 'slot'.
To achieve this, we use connection pools.
Connection pools are created with required concurrency limits and then assigned to connector instances. When one of these instances is configured in a process flow connector step, any requests are handled by the associated connection pool.
When a request is made, the connection pool limit is checked. If there's a free slot (i.e. the concurrency limit has NOT been reached), a slot is held and the request is sent. Once that request is complete, the slot is released. The illustration below shows how this works:
A single connection pool can be associated with a single instance; multiple instances of the same connector, or even multiple instances across different connectors.
Connection pools and rate limits work together. Rate limiting controls the number of API calls or requests that can be made within a set time frame (e.g. per minute), whereas connection pools manage the number of simultaneous connections allowed at any given time.
Suppose the rate limit for a connector instance allows up to 60 API calls per minute, but there's a concurrency limit of 10.
If you associate this connector instance with a connection pool configured with a limit of 10, only 10 connections can be active at once - even though the rate limit allows more requests per minute.
So, if all 10 connections are in use, additional calls will wait until a connection slot becomes available - even if you’re still under the rate limit. However, if the 10 concurrent connections are fast enough, you may hit your rate limit (60 calls per minute).
To access existing connection pools for your company profile, select the connection pools option from the left-hand navigation menu:
To create a new connection pool, follow the steps below:
Step 1 Access the connection pools page:
Step 2 Click the create button:
Step 3 Enter details for this connection pool:
Available settings are summarised below:
Name
Enter a name for this connection pool. This name is displayed wherever this pool is used in the system.
Limit
Enter the number of concurrent requests that can be made when this pool is used.
Timeout
If appropriate, enter the number of seconds that requests should wait for a connection pool slot to become available before timing out. The default timeout is 30 seconds - this is applied if nothing is defined here. The maximum timeout allowed is 300 seconds.
Step 4 Click the create button.
If you change the settings for an existing connection pool (for example, increase or decrease the limit), those changes are automatically picked up in any process flow connection steps that use this pool.
To update an existing connection pool, follow the steps below:
Step 1 Access the connection pools page:
Step 2 Click the name of the connection pool that you want to update:
Step 3 Update details as required, then click the update button:
There may be times when you need to clear requests from a connection pool - for example, if you're testing a process flow and want to start again.
If your connection pool is associated with live, production instances, use this option carefully. Clearing a connection pool clears all pending requests.
To clear a connection pool, follow the steps below:
Step 1 Access the connection pools page:
Step 2 Click the 'clear' icon associated with the connection pool you want to update:
Step 3 When prompted, confirm that you would like to clear all existing requests from this connection pool:
If necessary, you can delete an existing connection pool at any time. If you remove a connection pool that's associated with connector instances, it is removed from those automatically (so you would need to reconfigure your instances with alternative connection pools, if required).
Existing connection pools can be associated with connector instances when you add a new instance or update an existing instance. For example:
To remove a connection pool association from an instance, you should edit the instance and click the 'cross' icon for the existing connection pool. For example: