Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Some APIs include pagination functionality so that large HTTP responses can be served as multiple pages, which are more manageable. Different APIs use different methods to approach this.
Paginating requests for resources that might be large is recommended - otherwise the API may truncate the result or timeout whilst attempting to gather the data. Limiting requests to a smaller number of objects at a time typically results in a faster, more reliable outcome.
When paginated data is pulled, a payload is generated for each page.
To set a pagination method for an endpoint, click in the pagination method field to activate a dropdown list of supported methods:
Having selected a method, click save. Any required options for the selected method are displayed.
The following pagination methods are supported:
This method is similar to the approach, except that the URL for the next page is included in the response body, rather than the header.
| Option | Summary |
|---|
Suppose we set the following options:
| Option | Value |
|---|
...and we send a request to get the first page of data:
The response will include the first page of data, together with a URL that should be used to get the next page of results. For example:
Notice the links.next section at the end of this response, which includes our next page URL. So, our request for the next page of results would be:
Pagination continues until the next page URL is no longer included in the payload.
Next page field | Enter the location of the next page URL within the payload. |
Next page field |
|
This method is similar to the page number parameter approach. However, instead of appending a page number to the request URL, an offset parameter is used to indicate how many results have been returned so far (i.e. the start point for required data).
This method is rather like a database search: give me 5 records starting from the 15th record.
| Option | Summary |
|---|---|
Suppose we set the following options:
| Option | Value |
|---|---|
...and we send a request to get the first page of data:
The response will include the first page of data (sometimes with metadata regarding pagination). For example:
Requests for subsequent pages would increment the offset value by the limit value - for example:
...and then:
...and so on until 50 items (i.e. the total parameter path value) are returned.
Pagination continues until the last page, when there is no more data to return.
The limit offset pagination method is implemented using a Patchworks pagination variable which contains the next page offset:
This note is for reference only - there should never be a need to access/change this variable.
Each page is denoted by the ID of the last object it holds, rather than its page. For example, if a 'page' returns 10 customers, then the ID of the last customer will be passed in the URL to get the next page.
| Option | Summary |
|---|---|
Suppose we set the following options:
| Option | Value |
|---|---|
The following URL will be generated to get the first page:
Notice our link parameter path (limit) and the actual limit value (10) at the end of the request. Our first page would look something like the example below:
When pagination runs, it will look for the last id key in the payload, and use that as the starting point for the next request. So the request for page 2 will be:
Notice our link parameter path (limit) and the actual limit value (10) at the end of the request, followed by our Last ID parameter name (starting_after) set to the last id in the response.
A sample response for this request is below:
Pagination continues until the the number of items in the payload is less than the limit, or the page returns an error code.
GraphQL cursor pagination is a method for paginating GraphQL APIs.
Here, we send a GraphQL query requesting a list of items. This query includes parameters such as the number of items to fetch and any filtering or sorting criteria required. In response, requested items are received, together with cursor information (each item in the response is associated with a cursor, which is typically a unique identifier for that item).
The GraphQL cursor pagination method was developed for the Shopify GraphQL API however, the same principles apply for other GraphQL systems.
| Option | Summary |
|---|---|
The GraphQL cursor pagination method works in conjunction with a query on the endpoint body, which determines required settings. For example:
If you need to change the existing query for an endpoint, you should edit the connector and access body options for the required endpoint.
Pagination continues until hasNextPage is returned as false.
In this method, the required page number is included as a parameter in the API request, and is incremented in each request to get the next page.
| Option | Summary |
|---|---|
Suppose we set the following options:
| Option | Value |
|---|---|
...and we send a request to get the first page of data:
The response will include the first page of data (sometimes with metadata regarding pagination). For example:
Our request for the next page of results would be:
Pagination continues until the last page, when there is no more data to return.
The page number parameter pagination method is implemented using a Patchworks pagination variable which contains the next page number:
This note is for reference only - there should never be a need to access/change this variable.
This pagination method has been developed specifically for PeopleVox. It should not be used with other systems.
| Option | Summary |
|---|---|
PeopleVox GET endpoints include body content which handles specific requirements for PeopleVox data. This includes a Patchworks pagination variable, which is used to calculate page numbers:
This note is for reference only - there should never be a need to change this content.
This method is a slight variation on the approach. Instead of receiving a full URL in the response, it contains a 'token' (usually a random string/hash). The receiving system uses this token to keep track of the position of the last record in the current page of data.
| Option | Summary |
|---|
Suppose we set the following options:
| Option | Value |
|---|
...and we send a request to get the first page of data:
The response will include the first page of data, together with a next page token that should be used to get the next page of results. For example:
Notice the links.next section at the end of this response, which includes our next page URL. So, our request for the next page of results would be:
Pagination continues until the token is no longer included in the payload.
The next page token pagination method is implemented using a Patchworks pagination variable which contains a token for the next page:
This note is for reference only - there should never be a need to access/change this variable.
This pagination method has been developed specifically for the NetSuite SOAP connector. It should not be used with other systems.
| Option | Summary |
|---|
To find the required xPaths, tools such as can be useful. Paste in a sample NetSuite SOAP response (first page) to obtain the required paths to fields.
Link header name
Enter a name for the header that will contain link information. A commonly used name is: Link.
Limit parameter path
Enter the dot notation path for the data element in your response which contains the maximum number of items that the API should return in a single page of a paginated response. Commonly (though not necessarily), this would be a path to a limit field.
Offset parameter path
Enter the dot notation path for the data element in your response which contains the required offset - i.e. how many items should be skipped before the next page starts. Commonly (though not necessarily), this would be a path to a page field.
Limit
Enter the number of items to be returned per page. For example: 10.
Count parameter path
If required, enter the dot notation path for the data element in your response which contains the current count of records that have come back from the system. Commonly (though not necessarily), this would be a path to a count field. This value is not required for some APIs, therefore it's an optional field.
Total parameter path
Enter the dot notation path for the data element in your response which contains the total number of records that will be returned in this response, so we know when to stop paginating. Commonly (though not necessarily), this would be a path to a total field. This value is not required for some APIs, therefore it's an optional field.
Limit parameter path
limit
Offset parameter path
offset
Limit
5
Count parameter path
meta.count
Total parameter path
meta.total
Link parameter path
Enter the dot notation path for the data element in your response which contains the maximum number of items that the API should return in a single page of a paginated response. Commonly (though not necessarily), this would be a path to a limit field.
Last ID parameter name
Enter the name of the query parameter to be passed into the URL to tell the receiving system the last record we received (so the receiving system knows where to start for the next page of data to return). A commonly used name is: last_id .
Limit
Enter the number of items to be returned per page. For example: 10.
ID key
Enter the unique identifier key for each item in your data. For example, if the endpoint is pulling a list of users, this might be: user_id.
Link parameter path
limit
Last ID parameter name
starting_after
Limit
10
ID key
id
EndCursor path
Always defined in the following form:
data.{type}.pageInfo.endCursor
Here, the {type} element should be replaced with the type that's defined by the schema - for example:
data.products.pageInfo.endCursor
HasNextPage path
Always defined in the following form:
data.{type}.pageInfo.hasNextPage
Here, the {type} element should be replaced with the type that's defined by the schema - for example:
data.products.pageInfo.hasNextPage
Limit parameter name
Enter the name of the query parameter to be passed into the URL to tell the receiving system the maximum number of items that the API should return in a single page of a paginated response. A commonly used name is: limit but total or count are also popular.
Limit parameter path
Enter the dot notation path to the limit parameter name.
Page parameter name
Enter the name of the query parameter to be passed into the URL to tell the receiving system the required page number. A commonly used name is: page.
Limit
Enter the number of items to be returned per page. For example: 10.
Path to the data in the response
If required, you can add a dot notation path to an element in the payload that you would expect to contain data if the data pull has not completed. For example, if you are pulling orders, you might expect to see data in an orders element of the response.
This is a 'belts and braces' option to handle edge cases where a system continues to send data of some type, even when there is no more required data to be pulled - so pagination would never stop. If you don't want to use this option, click the allow empty button.
Limit parameter name
limit
Page parameter name
page
Limit
10
Path to the data in the response
data
Limit
Enter the number of items to be returned per page. For example: 50.
Total count tag field
The name of the PeopleVox response field which contains the total number of records available. For PeopleVox this is TotalCount.
Token parameter name | Enter the name of the URL parameter which holds the token for requesting the next page. |
Limit parameter path | Enter the dot notation path for the data element in your response which contains the maximum number of items that the API should return in a single page of a paginated response. Commonly (though not necessarily), this would be a path to a |
Limit | Enter the number of items to be returned per page. For example: |
Path to token in payload | Enter the dot notation path of the next page token within the payload. |
Token parameter name |
|
Limit parameter path |
|
Limit |
|
Path to token in payload |
|
Limit | Enter the number of items to be returned per page. For example: |
XPATH to the search ID field in the response | Enter the XPath to the ID field in your request responses. |
XPATH to the total pages field in the response | Enter the XPath to the total pages field in your request responses. |