Webscrape one then increment one then scrape again then increment again and scrape again

How To
1

I am trying to build a scraper that will search the fmcsa database of new applicants for a usdot number. The problem I am having is figuring out how to get the usdot number to increase by one and run again cause I need the full list of usdot numbers created yesterday. I have tried a couple of different scenarios now but am completely stuck, the fmcsa data base will only return info on one USDOT number at a time. I have tried set variable, increment but I am only getting the fmcsa address etc. back when attempting. I had hoped to make use of the repeater to just make it fire off fifty times then chill for a bit then 50 again but I cant figure out how to automatically increase the USDOT number by 1 and run again.

2

I am little confused. Are you looking solution for scenario or for custom app??

If you are looking for custom app then you need to use pagination in your module communication so app cab directly call in background.

If you are looking for scenario help, then show some screenshots.

3

Welcome to the Make community!

To do this, you can try using the Flow Control “Repeater” module.

For more information about the Repeater module and Flow Control app, see the corresponding Integrations page and the Help Centre documentation.

Here are some useful links and guides you can use to learn more on how to use the Make platform, apps, and app modules. I found these useful when I was learning Make, and hope they might benefit you too —

Getting Started

Help Centre Basics

Articles & Videos

Hope this helps! Let me know if there are any further questions or issues. P.S.: investing some effort into the tutorials in the Make Academy will save you lots of time and frustration using Make!

— @samliew