Retrieving values from deeply nested collections

[calebsg]

Make noob here – moving over from Zapier. I am having trouble with highly nested collections where I want to retrieve data into a variable. I am using the Set Variable tool but not sure how to use the map or get formulas when needing to go down a few layers.

For example, I would like a variable called “Client Address” to have the value of “Value” under Field ID 4088061. And I would like a variable called “First Time” to have the value of “Value” under Field ID 5035053. The Field IDs are consistent across all records.

The formula I have tried is {{map(3.forms; “value”; “fieldID”; 5035053)}} and it returns empty.

TIA!

[alex.newpath]

@calebsg

map(3.forms; “value”; “fieldID”; 5035053)

This is much more complex than one appreciates. The forms array as collections inside so you’ll need to iterate the array first to extract in each collection the objects you want. Make a new array with these with an array aggregator and only then use the map() function on the new array you created with that values you need.

[Robert_McKay]

Oof. Hi Alex. It’s been a minute since you onboarded me with Make (what a great intro to the product). And while I’ve come a ways, I’m always finding things like this I didn’t know!

So I think this thread would explain why I’m not matching on email here:

The Email key is three levels down, so the following map function isn’t finding it, right?

I tried putting in an iterator and aggregator to get just the emails, but ran into a weird behavior I hadn’t seen where the iterator only returns the first bundle. Does it always do that if you map a sub-item instead of the top-level array?

As an aside, does the placement of the lower() functions look right to you, in terms of making both what the map() function returns and the filtering key be lowercase, so that if someone types their email with weird caps it will still match?

[alex.newpath]

Hi there. I’m not sure you are getting what map is for. The arguments to the function map() are an array and a key in the array. You don’t pass in the values unless you’re filtering using the third and fourth argument.

So to get all the emails you will use something like this

Map(6.array;custom_fields)

This will return all the email keys from custom_fields in an array.

An iterator module simply iterates the array you pass into it and runs the next set of modules with the value of each array element. Bundles will naturally be processed as bundles always are, one at a time by the module that generates the output bundles. Each output bundle will then run all the modules and the bundle contents are available individually to all the modules after the module generating the bundles.

Think of bundles like processing a row of data with nested values possibly as arrays and collections. But in each bundle you may have arrays that you can process with an iterator (generating output bundles for each element of the array) or use map() function if you want to just manipulate the array and extract parts of it for use in your scenario.

[Robert_McKay]

Ah, thanks. I think I was unclear about whether & how I could use dot notation to drill down. Will try that.

[alex.newpath]

Here’s how I like to experiment with structures. I use Parse JSON module to model the output bundles I expect to get. So I modeled it with this JSON:

{

"Array": [
        {
            "id": "866a7nubx",
            "name": "Testy McTest",
            "custom_fields": [
                {
                    "Email": "testy@mctest.com",
                    "lastappointment": 1234,
                    "touchpoint": [{
                                    "a": "abc",
                    "b": "bcd",
                    "c": 123} ]
                }
            ]
        },
 {
            "id": "860qtdae",
            "name": "Testy2 McTest2",
            "custom_fields": [
                {
                    "Email": "testy2@mctest.com",
                    "lastappointment": 1234,
                    "touchpoint": [{
                                    "a": "abc",
                    "b": "bcd",
                    "c": 123} ]
                }
            ]
        }
  ]
}

The output of Parse JSON is exactly what you’d get out of your regular module:

Here’s how to iterate on the custom_fields array:

And since the array had 2 elements in it, the iterator generates 2 bundles:

Now you can place a set variable and each run through the iterator you will be able to grab the email address from each bundle. This syntax allows you to grab the output from the iterator

Notice there are 2 operations on set variable since the array had 2 elements

Now if you want to combine the output of the bundles from set variable into 1 array with each email in it then you need to array aggregator with the source module being the iterator and click the email from set variable in your aggregator

the result will have 1 array with both emails in it

You can of course place a filter between the Set Variable and the Array Aggregator and only grab the values you want (by using a filter on email address). This will only aggregate into an array any variables set that meet your filter criteria.

Any modules after array aggregator will only execute once since the array aggregator created just 1 bundle (with the array). All aggregators combine multiple bundles into 1 array, by the way.

[Robert_McKay]

Hey Alex, this is conceptually very helpful; appreciate you taking the time. I think for my use case all I need is a solution to the case problem.
This new version of the test run is working, drilling down with dot notation to get the email out of the custom fields.

It can also get items other than the first one in the array, without using iterators and aggregators to make an array just of that one item.

So it seems like this method should work to check if any of the array items match what’s coming in. The only problem is when the incoming payload has different capitalization than what’s in the array I’m checking against.

And wrapping the key and filter in lower() breaks it!

Wondering if there’s some other way to arrange the functions so you can apply lower case to the array items before comparing them to the lower-cased version of the incoming payload. If not, I guess I’ll have to use a route filter that ignores case, which would also necessitate iterating through the array against which I’m checking, so that each bundle of existing data is checked one by one by the route filter against the payload of new data.

Or is there a more elegant way to do it?

[alex.newpath]

Why do you need to lowercase the third argument name? Isn’t that the key you’re filtering against? Just keep the the lower on the last argument.

[Robert_McKay]

It’s the key but I was trying to make the array contents the map function returns be lower case, as well as the last argument against which I’m checking them. Otherwise if a kid types their name all lowercase, as they often do, and I have their name in the CRM (where the array is coming from) as “McTest,” it won’t match. Putting what they typed in start case won’t solve it here either if you have a capital within the word. Hence the attempt to lower case both be array and what I’m matching against.

[alex.newpath]

Aha I see what you mean now. I think you’ll need an intermediate module to make that work with set multiple variables to lower case your emails, then you can use them to elsewhere. The lower() string function can be used on individual strings and never on arrays.

[vendy]

Hey there @calebsg

just wanted to step in and ask if you find a solution for your struggle. If yes, could you share it here with us?

This way our community stays organized for other users.

Thank you

[calebsg]

Thanks, Alex – sorry for the very belated response but this worked well for me!