I need to access data in nested collections and arrays

[jeroenvanderwielen]

What is your goal?

Extract the price, title and name of certain scraped products from Google that are nested in an array with nested collections and arrays.

What is the problem & what have you tried?

I tried to use combinations of get and map functions. I think I am on the right track but haven’t found the combination yet (or I am completely on the wrong track).

I have the same problem with a simpler nested array of collections, which I will upload as well.

Error messages or input/output bundles

I will provide an example of the output bundles of the array aggregators. My goal is to map the yellow marked fields in a Google Sheets.

Screenshots (scenario flow, module settings, errors)

[Stoyan_Vatov]

Hey there,

you do need the get(map()) combo to find it yeah. But since its two arrays, you will need to stack it.

Can you share the complete JSON you are trying to find the data in and we can help build the exact formula?

[jeroenvanderwielen]

Hi, Thanks for your fast response.

This is the JSON:
[
{
“array”: [
{
“backlink_check”: {
“domainRating”: 44,
“backlinks”: 3322,
“refdomains”: 1002,
“dofollowBacklinks”: 100,
“dofollowRefdomains”: 100,
“urlRating”: 9,
“ahrefsRank”: 1070008
}
},
{
“backlink_check”: {
“domainRating”: 92,
“backlinks”: 0,
“refdomains”: 0,
“dofollowBacklinks”: 0,
“dofollowRefdomains”: 0,
“urlRating”: 0,
“ahrefsRank”: 400
}
},
{
“backlink_check”: null
},
{
“backlink_check”: {
“domainRating”: 1.1,
“backlinks”: 25,
“refdomains”: 8,
“dofollowBacklinks”: 72,
“dofollowRefdomains”: 13,
“urlRating”: 5,
“ahrefsRank”: 50921934
}
},
{
“backlink_check”: {
“domainRating”: 67,
“backlinks”: 1,
“refdomains”: 1,
“dofollowBacklinks”: 100,
“dofollowRefdomains”: 100,
“urlRating”: 4,
“ahrefsRank”: 141093
}
},
{
“backlink_check”: null
},
{
“backlink_check”: {
“domainRating”: 34,
“backlinks”: 0,
“refdomains”: 0,
“dofollowBacklinks”: 0,
“dofollowRefdomains”: 0,
“urlRating”: 4,
“ahrefsRank”: 2614111
}
},
{
“backlink_check”: {
“domainRating”: 26,
“backlinks”: 0,
“refdomains”: 0,
“dofollowBacklinks”: 0,
“dofollowRefdomains”: 0,
“urlRating”: 0,
“ahrefsRank”: 5585036
}
}
],
“_IMTAGGLENGTH_”: 8
}
]

[jeroenvanderwielen]

This is my current setup btw, It fills the google sheets with N/A

[Stoyan_Vatov]

Ok but do you need all of them or only specific ones? Cause it looks like its an array of similar collections.

[jeroenvanderwielen]

Hi, Yes so it’s usually an array of length 5-10 (depending on how many products are found). Then in every collection inside the array, the data structure is the same. Like this in the picture:

[Stoyan_Vatov]

And what are you trying to do with the data? Send it all together somewhere or do something with each different entry?

[jeroenvanderwielen]

I want to upload it all in Google Sheets:

You can see all the N/A’s are because the combination of get() and map() that I currently use returns an empty array probably. I need them filled for all products (8 in this case), but like I said, the range is 5-10 products, so once I have the correct formula, I just copy paste it in the Google Sheets ‘Update a Row’ module and replace the number in the get() function with the number of the product

[Stoyan_Vatov]

But you want each item in a new row no? get(map()) is meant to be used when you need to find specific items from the array. Here it sounds like you just need to process all of them. So use an iterator instead and create a new row in the sheet for each item.

[jeroenvanderwielen]

No, they are all in the same row for certain keywords. But I think I already found the solution, I will show when I’m done

[Msquare_Automation]

Hi @jeroenvanderwielen ,

To add the fields to the Google Sheet in your case, the recommended approach is to use the bulk add row method. This allows you to insert multiple rows at once efficiently, instead of mapping fields one by one.

I have prepared a sample scenario based on your target setup and am attaching it here. It demonstrates how to structure the data, map the fields, and execute the bulk addition. Please review it carefully and try running it with your data.
[COMMUNITY] -TEST.blueprint(4).json (10.8 KB)

If you encounter any issues, whether with field mapping, scenario execution, or data formatting, please don’t hesitate to reach out. I’ll be happy to guide you through any challenges and ensure it works smoothly.

Looking forward to your feedback!

Best regards,
Msquare Automation
Platinum Partner of Make
@Msquare_Automation

[Stoyan_Vatov]

Ok you only need map() then to create a primitive array of each value. Then use join() on top of it to create a comma separated list.

[jeroenvanderwielen]

So getting back: I’ve figured out how to do it. In this case, get(get(map({{array}};backlink_check);1);domainRating) = 44,

get(get(map({{array}};backlink_check);2);domainRating) = 92. And so on…

I found out there are multiple ways to do so, for example:
get(get(get({{array}};1);backlink_check);domainRating) also would return 44 as output here.

The problem for now is thus solved, but a question remains when I only want to get the domainRating where refdomains = 30 for example. This would be easily done with map, but since there are collections here, it is a little but different.