Hi everyone,
I’m facing the following challenge: my API is limited to 1000 records. When total records exceeds that like 2149 as shown in the image, how can I append the records so they don’t get cut off at 1000?
I’ve tried using the API parameters limit and offset with the “Data Store” module.
I’m open to more efficient approaches, possibly involving loops.
If you have any questions, feel free to ask!
Hey there,
you will have to check the API documentation and how it handles pagination.
From what I can tell, you should be able to use offset to get the next 1000 records and then again for the last 149. What we usually do is use a data store to record the page we are on and then keep retriggering the scenario until there are no more pages to process.
Or you can use one call to see how many records there are in total and then use a formula inside a repeater module to keep incrementing the offset until everything has been processed.
