I am building an automation to extract the HTML content from some URLs.
The automation steps are as follows: I add a specific term to Google sheet, then I search for it in the Google results, then I get the list of URLS, I extract the HTML from those URL, I convert it to txt and so on.
Problem is: I expected it to fetch the HTML content once per page, but instead, I’m getting multiple HTML extractions for the same pages. So for example, last time I ran it I had a list of 7 URL but more than 30 operations during the HTML extraction.
I already tried to debug it and broke the automation in chunks to see where it went wrong:
· I hardcoded the URL to get the HTML from and that worked fine: 1 URL in 1 URL out
· I connected my sheet to the HTML module (so URL was not hardcoded this time) and that worked fine: 1 URL in 1 URL out
· I then connected the rest of the automation: searched for the term in Google, added the URLs I got to the spreadsheet, passed it to HTML module, and that where it went wrong: I got 5 URLs, but way more HTML conversions, and I don’t understand why.
Here are the screenshots. In the second one you can see where it goes wrong. Actually it starts going wrong in the HTTP module but I wanted to take a screenshot with the actual numbers
Based on your screenshot, your Iterator is outputting 4 bundles. Is this what you expected?
Also, your “Search Rows” module outputs a bundle for EACH result for EACH bundle before it. Are you expecting a single result? If not, you should be aggregating those search “result” rows.
Combining Bundles Using Aggregators
Every result (item/record) from trigger/iterator/list/search/match modules will output a bundle. This can result in multiple bundles, which then trigger multiple operations in future modules (one operation per bundle). To “combine” multiple bundles into a single variable, you’ll need to use an aggregator of some sort.
Aggregators are modules that accumulate multiple bundles into one single bundle. An example of a commonly-used aggregator module is the Array aggregator module. The next popular aggregator is the Text Aggregator which is very flexible and can apply to many use-cases like building of JSON, CSV, HTML.
You can find out more about the other types of aggregator modules here:
No problem, glad I could help you with “More operations then expected in HTML extraction”
1. If anyone has a new question in the future, please start a new thread. This makes it easier for others with the same problem to search for the answers to specific questions, and you are more likely to receive help as newer questions are displayed higher on the forum’s “new” page.
2. The Make Community guidelines encourages users to try to mark helpful replies as solutions to help keep the Community organized.
This marks the topic as solved, so that:
others can save time when catching up with the latest activity here, and
allows others to quickly jump to the solution if they come across the same problem
To do this, simply click the checkbox at the bottom of the post that answers your question:
3. Don’t forget to like and bookmark this topic so you can get back to it easily in future!
Here are some useful links and guides you can use to learn more on how to use the Make platform, apps, and app modules. I found these useful when I was learning Make, and hope they might benefit you too —
How to schedule scenario for first Monday next month (recurring)
Automate Follow-Up of Email Leads a Week After