I have this scenario that scrapes websites, uses a regex to take data, then updates a row in a Google sheet.
The problem is, the HTTP module works slower than the update a row module, and it keeps running over and over again while it’s waiting for more data. I’ll end up with say, 50 operations for the HTTP module, 50 for the regex match pattern, then 150 update rows.
Is this normal behaviour? I can’t use an aggregator or iterator as I won’t be able to access the data inside that loop, which is needed for the Google Sheets module.
Is there a solution?
Yes, this is normal.
Only one bundle can be processed at the same time (in parallel).
The only other way is to aggregate all the bundles into a single array, then process the array using the array functions, but in this case you can’t do that with the HTTP module.
For more information, see
Here are some useful links and guides you can use to learn more on how to use the Make platform, apps, and app modules. I found these useful when I was learning Make, and hope they might benefit you too —
General
Help Center Basics
Articles & Videos
samliew – request private consultation
Join the Make Fans Discord server to chat with other makers!
1 Like
