API only returns 1000 records at a time

How To
1

I have an API for an industry-specific CRM.

To pull complete customer data, I must first make an API call that returns all the customer IDs.

Once I have the list of IDs, I can make another API call with up to 1000 customer IDs at a time that will return the complete customer details for those 1000 IDs.

Is there a way to iterate through the original list of customer IDs and make multiple API requests to get all the customer details?

Thanks for any insights!

2

Hello @TMGIII,

Please see these resources on the subject of pagination in Make:

https://community.make.com/t/master-pagination-techniques-of-api-in-make-formerly-integromat/7140

Edit: Removed other link as it wasn’t related to Make.

2 Likes
3

It boils down to something like this:

  • Make your initial HTTP request and use the results to figure out either how many pages of results there are to capture, or how many records there are, along with the query parameters needed to capture groups of records.
  • Use a repeater to start a loop. You’ll need to use the info from the previous HTTP module to figure out where to start and end
  • Use an HTTP module after the repeater. Use the repeater’s output value of {{i}} in this HTTP module to make each call to the API unique (get either a different page or different group of results each time)
  • Follow with an Iterator and Text or Array Aggregator to aggregate the results of each HTTP call.

A good resource is in Make Partner Training, but you’ll need to sign up to view it: https://partnertraining.make.com/courses/take/make-level-4-advanced-training/lessons/41778722-27-video

2 Likes