Integromat is limiting my following scraping to 100

Questions & Answers
,
1

image
image1588×471 46.8 KB

So actually I’m trying to scrape data with airtable and integromat and but when I’m trying to scrape 1, 2, 3 it just stacks at 100, 200, 300 meaning that scrapes just 100 following from a page and it’s not from the API because I’ve talked with the developer and his api doesn’t stuck at one point.

I am looking for solution for following scenario
I’ll make an HTTP request & response is in json format always, if response contains next_max_id key then I need to make another http request (with next_max_id as query parameter) untill response doesn’t contains next_max_id

2

Does this solution in this post helps with your need?

The current problem that you have is Make doesn’t auto-iterate based on next_max_id in the HTTP response, So it will always be 100 results and since you are not iterating over HTTP it will always end with single execution thus resulting in 100 results only.

1 Like
3

I’ll check it out, thanks

4

Heya @zarely :wave:

I was wondering if you’ve found a minute to check out @Runcorn’s suggestion and made it work. If yes, could you mark it as a solution? :white_check_mark:

This process keeps our community clean and tidy for others to look for solutions. Thanks in advance!