Hi,
I created an automation to scrape my competitor website using Apify WebScraper. Setup is Webhook, Apify Get Dataset Items, create a spreadsheet, then add row (titles, url, etc), then add multiple row to fillup the spreadsheet with the result from WebScraper.
Everything is working except for adding all the result in spreadsheet.
What do I map in the rows?
When mapping the rows in your spreadsheet, ensure that you are correctly linking the extracted dataset fields from Apify WebScraper to the corresponding columns in Google Sheets. Typically, you need to reference the dataset items dynamically rather than as a single object. If the extracted data is an array, use a looping mechanism or batch processing to add each item as a separate row. Check that the “Add Multiple Rows” step is set to accept an array and not just a single entry. If possible, share the JSON structure of the dataset to provide more precise guidance.
Here is a sample JSON [ { "contestName": "5th Vincent van Gogh Photo Award 2025", "deadline": "30 March 2025", "contestLink": "http://www.vincentvangoghphotoaward.com", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/5th-vincent-van-gogh-photo-award-2025/", "__IMTLENGTH__": 10, "__IMTINDEX__": 1 }, { "contestName": "CEWE Photo Award 2025", "deadline": "30 May 2025", "contestLink": "https://contest.cewe.co.uk/cewephotoaward-2025/en_gb/", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/cewe-photo-award-2025/", "__IMTLENGTH__": 10, "__IMTINDEX__": 2 }, { "contestName": "200/20 Binghatti Photography Competition", "deadline": "28 March 2025", "contestLink": "https://art.binghatti.com/", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/200-20-binghatti-photography-competition/", "__IMTLENGTH__": 10, "__IMTINDEX__": 3 }, { "contestName": "Chromatic Photo Awards 2025", "deadline": "20 July 2025", "contestLink": "https://chromaticawards.com/", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/chromatic-photo-awards-2025/", "__IMTLENGTH__": 10, "__IMTINDEX__": 4 }, { "contestName": "2025 MUSE Photography Awards", "deadline": "28 February 2025", "contestLink": "https://musephotographyawards.com/", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/2025-muse-photography-awards/", "__IMTLENGTH__": 10, "__IMTINDEX__": 5 }, { "contestName": "Street Photography Contest 2025 by The Artist Gallery Awards", "deadline": "6 March 2025", "contestLink": "http://www.theartistgallery.art/open-contests", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/street-photography-contest-2025-by-the-artist-gallery-awards/", "__IMTLENGTH__": 10, "__IMTINDEX__": 6 }, { "contestName": "Chiaroscuro: The Art of Light and Shadow", "deadline": "31 March 2025", "contestLink": "https://www.decagongallery.com/chiaroscuro-the-art-of-light-and-shadow-entry", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/chiaroscuro-the-art-of-light-and-shadow/", "__IMTLENGTH__": 10, "__IMTINDEX__": 7 }, { "contestName": "2025 Andrei Stenin International Press Photo Contest", "deadline": "28 February 2025", "contestLink": "https://stenincontest.com/", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/2025-andrei-stenin-international-press-photo-contest/", "__IMTLENGTH__": 10, "__IMTINDEX__": 8 }, { "contestName": "Wiki Loves Folklore 2025", "deadline": "31 March 2025", "contestLink": "https://commons.wikimedia.org/wiki/Commons:Wiki_Loves_Folklore_2025", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/wiki-loves-folklore-2025/", "__IMTLENGTH__": 10, "__IMTINDEX__": 9 }, { "contestName": "Monochrome Awards 2025", "deadline": "6 July 2025", "contestLink": "https://monoawards.com", "moreDetails": "https://photocontestdeadlines.com/photo-contest-list/monochrome-awards-2025/", "__IMTLENGTH__": 10, "__IMTINDEX__": 10 } ]
Pretty new with Make. Can you suggest what to use for the looping mechanism or batch processing? I want to fill 1 bundle per row.
Welcome to the Make community!
When you see an ARRAY, think ITERATOR module.
Then,
Every result (item/record) from trigger/iterator/list/search/match modules will output a bundle. This can result in multiple bundles, which then trigger multiple operations in future modules (one operation per bundle). To “combine” multiple bundles into a single variable, you’ll need to use an aggregator of some sort.
Aggregators are modules that accumulate multiple bundles into one single bundle. An example of a commonly-used aggregator module is the Array aggregator module. The next popular aggregator is the Text Aggregator which is very flexible and can apply to many use-cases like building of JSON, CSV, HTML.
You can find out more about the other types of aggregator modules here:
The Array Aggregator module is very powerful because it allows you to build a complex array of collections for a later module’s field to map multiple items (collections) to it.
This is done using the “Target structure type” field in an Array Aggregator module.
Here is an example:
As you can see, the “Map” toggle on complex fields are used when you have an array. You can easily build an array variable to map to a future module’s field, by using an Array Aggregator module and select the “Target Structure Type” as the future module’s field you have mapped the array into.
Here is an example of how your scenario could look:
This is just an example. Your final solution may or may not look like this depending on your requirements.
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.
This is awesome @samliew
Thank you! This helped alot.
