Get Values From Grouped Array Aggregator Output

[alexs]

I’ve got an array aggregator which gives me a list of IDs, broken down by category

like:

[
    {
        "__IMTKEY__": "Preference",
        "array": [
            {
                "id": "32b5edf9-b8a6-426e-b860-7e7116dd2b10"
            }
        ]
    },
    {
        "__IMTKEY__": "Sick",
        "array": [
            {
                "id": "32b5edf9-b8a6-426e-b860-7e7116dd2c13"
            }
        ]
    },
    {
        "__IMTKEY__": "Holiday",
        "array": [
            {
                "id": "c41f69ae-e542-4095-934d-04511699aaad"
            }
        ]
    }
]

Now I want to fetch all of the IDs from the Sick and Holiday groups and combine them into an array, stored as an “Unavailable” variable and the IDs from the Preference (& maybe other groups) and combine them into an array called “Preference”.

I can’t figure out the most efficient way to do this?

Make shows that the array aggregator output is an array but it looks like I can’t just map it? Even if I iterate through the bundles, I can think of a simple way to combine the IDs from more than 1 array into a list that’s stored as a particular variable name.

Here’s a blueprint for testing -

blueprint (19).json (4.7 KB)

[Mr.Make]

I was having an issue mapping Holiday and Sick together in one operation using one function so it took two for some reason. But final solution in the same. Can definitely be optimized but if you need a fix now this will solve it. I added the blue print. Also your blue print made it super easy to trouble shoot wanted to say nice job there :).

blueprint (7).json (11.5 KB)

[Mr.Make]

If you want to do other groups you can just use the same logic I used in getting the id for Holiday and Sick groups then just merge the arrays.

[alexs]

@Mr.Make

Also your blue print made it super easy to trouble shoot wanted to say nice job there :).

Glad that was useful

Thanks a lot for this, I’m going to hold out in the hopes that there’s a more efficient solution, in terms of operations used but it’s great to have a way to do this

[Mr.Make]

Sounds good let see what other people solve it. I think best case you can save one operations but we will see. Good luck.

[alex.newpath]

Here’s another way

I set a variable to extract each ID from each array in each bundle before I aggregate the original set of bundles. Then use the map() to extract the IDs from the various __IMTKEY___ names. Finally I use flatten() which converts multiple arrays into 1. This will allow multiple set of IDs for each original bundle if that is a possibility.

Module Export

You can copy and paste this module export into your scenario. This will paste the modules shown in my screenshots above.

1. Expand the “View Scenario JSON Blueprint” viewer and click the copy button when you mouseover the top-right of the code block
   
2. Enter your scenario editor. Press ESC to close any dialogs. Paste your clipboard (eg CTRLV) to add the blueprint code directly into the Make scenario canvas. Each module should appear in your canvas.
3. Click on each imported module and save it for validation. You may be prompted to remap some variables or select some connections before you can run the scenario.

View Scenario JSON Blueprint

{
    "subflows": [
        {
            "flow": [
                {
                    "id": 3,
                    "module": "json:ParseJSON",
                    "version": 1,
                    "parameters": {
                        "type": ""
                    },
                    "mapper": {
                        "json": "[\n    {\n        \"__IMTKEY__\": \"Preference\",\n        \"array\": [\n            {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2b10\"\n            },\n {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2b11\"\n            }\n        ]\n    },\n    {\n        \"__IMTKEY__\": \"Sick\",\n        \"array\": [\n            {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2c13\"\n            }\n        ]\n    },\n    {\n        \"__IMTKEY__\": \"Holiday\",\n        \"array\": [\n            {\n                \"id\": \"c41f69ae-e542-4095-934d-04511699aaad\"\n            }\n        ]\n    }\n]"
                    },
                    "metadata": {
                        "designer": {
                            "x": 0,
                            "y": 0
                        },
                        "restore": {
                            "parameters": {
                                "type": {
                                    "label": "Choose a data structure"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "type",
                                "type": "udt",
                                "label": "Data structure"
                            }
                        ],
                        "expect": [
                            {
                                "name": "json",
                                "type": "text",
                                "label": "JSON string",
                                "required": true
                            }
                        ]
                    }
                },
                {
                    "id": 18,
                    "module": "util:SetVariable2",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "name": "id",
                        "scope": "roundtrip",
                        "value": "{{map(3.array; \"id\")}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 300,
                            "y": 0
                        },
                        "restore": {
                            "expect": {
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "name",
                                "type": "text",
                                "label": "Variable name",
                                "required": true
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            },
                            {
                                "name": "value",
                                "type": "any",
                                "label": "Variable value"
                            }
                        ],
                        "interface": [
                            {
                                "name": "id",
                                "label": "id",
                                "type": "any"
                            }
                        ]
                    }
                },
                {
                    "id": 17,
                    "module": "builtin:BasicAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 3
                    },
                    "mapper": {
                        "id": "{{18.id}}",
                        "__IMTKEY__": "{{3.`__IMTKEY__`}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 600,
                            "y": 0
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "JSON - Parse JSON [3]"
                                },
                                "target": {
                                    "label": "Custom"
                                }
                            }
                        }
                    }
                },
                {
                    "id": 19,
                    "module": "util:SetVariables",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "variables": [
                            {
                                "name": "Unavailable",
                                "value": "{{flatten(map(17.array; \"id\"; \"__IMTKEY__\"; \"Sick,Holiday\"))}}"
                            },
                            {
                                "name": "Preference",
                                "value": "{{flatten(map(17.array; \"id\"; \"__IMTKEY__\"; \"Preference\"))}}"
                            }
                        ],
                        "scope": "roundtrip"
                    },
                    "metadata": {
                        "designer": {
                            "x": 900,
                            "y": 0
                        },
                        "restore": {
                            "expect": {
                                "variables": {
                                    "items": [
                                        null,
                                        null
                                    ]
                                },
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "variables",
                                "type": "array",
                                "label": "Variables",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Variable name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Variable value",
                                        "type": "any"
                                    }
                                ]
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            }
                        ],
                        "interface": [
                            {
                                "name": "Unavailable",
                                "label": "Unavailable",
                                "type": "any"
                            },
                            {
                                "name": "Preference",
                                "label": "Preference",
                                "type": "any"
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}

[alex.newpath]

I just realized you can just aggregate the original array without doing anything to it and then use a combination of 2 map() and a flatten to get the desired output.

I added another collection with an id to the Preference just to test it

Module Export

You can copy and paste this module export into your scenario. This will paste the modules shown in my screenshots above.

1. Expand the “View Scenario JSON Blueprint” viewer and click the copy button when you mouseover the top-right of the code block
   
2. Enter your scenario editor. Press ESC to close any dialogs. Paste your clipboard (eg CTRLV) to add the blueprint code directly into the Make scenario canvas. Each module should appear in your canvas.
3. Click on each imported module and save it for validation. You may be prompted to remap some variables or select some connections before you can run the scenario.

View Scenario JSON Blueprint

{
    "subflows": [
        {
            "flow": [
                {
                    "id": 3,
                    "module": "json:ParseJSON",
                    "version": 1,
                    "parameters": {
                        "type": ""
                    },
                    "mapper": {
                        "json": "[\n    {\n        \"__IMTKEY__\": \"Preference\",\n        \"array\": [\n            {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2b10\"\n            },\n {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2b11\"\n            }\n        ]\n    },\n    {\n        \"__IMTKEY__\": \"Sick\",\n        \"array\": [\n            {\n                \"id\": \"32b5edf9-b8a6-426e-b860-7e7116dd2c13\"\n            }\n        ]\n    },\n    {\n        \"__IMTKEY__\": \"Holiday\",\n        \"array\": [\n            {\n                \"id\": \"c41f69ae-e542-4095-934d-04511699aaad\"\n            }\n        ]\n    }\n]"
                    },
                    "metadata": {
                        "designer": {
                            "x": 0,
                            "y": 0
                        },
                        "restore": {
                            "parameters": {
                                "type": {
                                    "label": "Choose a data structure"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "type",
                                "type": "udt",
                                "label": "Data structure"
                            }
                        ],
                        "expect": [
                            {
                                "name": "json",
                                "type": "text",
                                "label": "JSON string",
                                "required": true
                            }
                        ]
                    }
                },
                {
                    "id": 22,
                    "module": "builtin:BasicAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 3
                    },
                    "mapper": {
                        "__IMTKEY__": "{{3.`__IMTKEY__`}}",
                        "array": "{{3.array}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 300,
                            "y": 0
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "JSON - Parse JSON [3]"
                                },
                                "target": {
                                    "label": "Custom"
                                }
                            }
                        }
                    }
                },
                {
                    "id": 21,
                    "module": "util:SetVariables",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "variables": [
                            {
                                "name": "Unavailable",
                                "value": "{{map(flatten(map(22.array; \"array\"; \"__IMTKEY__\"; \"Sick,Holiday\")); \"id\")}}"
                            },
                            {
                                "name": "Preference",
                                "value": "{{map(flatten(map(22.array; \"array\"; \"__IMTKEY__\"; \"Preference\")); \"id\")}}"
                            }
                        ],
                        "scope": "roundtrip"
                    },
                    "metadata": {
                        "designer": {
                            "x": 600,
                            "y": 0
                        },
                        "restore": {
                            "expect": {
                                "variables": {
                                    "items": [
                                        null,
                                        null
                                    ]
                                },
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "variables",
                                "type": "array",
                                "label": "Variables",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Variable name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Variable value",
                                        "type": "any"
                                    }
                                ]
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            }
                        ],
                        "interface": [
                            {
                                "name": "Unavailable",
                                "label": "Unavailable",
                                "type": "any"
                            },
                            {
                                "name": "Preference",
                                "label": "Preference",
                                "type": "any"
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}

[alexs]

Thanks a lot for this! Seems crazy that you have to add an array aggregator after an array aggregator to get the output into the correct format so I’m very grateful that you figured that out!

[alex.newpath]

@alexs

Seems crazy that you have to add an array aggregator after an array aggregator to get the output into the correct format

I am not sure you have to do that actually – I was working with the JSON snippet for the structure you gave me and the Parse JSON module de-aggregates the aggregated array so that’s why I reaggregated it.

You can put the expressions right after the first aggregator.

[alexs]

Oh ok, the JSON snippet was the output of an array aggregator which created those groups

[alex.newpath]

Right and I used the Parse JSON to convert that to multiple bundles again so I had to reaggregate it. You can put the set variable expressions I used or just adjust the expressions to reference the array coming out of your array aggregator directly.

[alexs]

@alex.newpath

Right and I used the Parse JSON to convert that to multiple bundles again so I had to reaggregate it.

Ah, I didn’t realise that.

@alex.newpath

You can put the set variable expressions I used or just adjust the expressions to reference the array coming out of your array aggregator directly.

The thing that tripped me up there - and the reason why I couldn’t get this working to begin with - is that it seems as though it’s not possible to reference the raw output of the array aggregator module as an array. When I’m building the next step, the UI shows this -

Even though the output of step 6 is an array containing collections containing the keys and nested arrays -

[
    {
        "__IMTKEY__": "Preference",
        "array": [
            {
                "id": "14ee157f-98bf-4f70-b092-eea9eaa759e1"
            }
        ]
    },
    {
        "__IMTKEY__": "Sabbatical",
        "array": [
            {
                "id": "32b5edf9-b8a6-426e-b860-7e7116dd2b10"
            }
        ]
    },
    {
        "__IMTKEY__": "Holiday",
        "array": [
            {
                "id": "c41f69ae-e542-4095-934d-04511699aaad"
            }
        ]
    }
]

I guess Make’s converting the top level array into bundles because it doesn’t have a name -

As a result, I couldn’t see how to map that top level array?

[Msquare_Automation]

Hey @alexs

From array, you will see only first item while mapping. However, you will need to use get and map function to fetch the desired values from the array.

If you require additional assistance, please don’t hesitate to reach out to us.
MSquare Support | Book Live Implementation
Visit us here
Youtube Channel

[alexs]

Right but in order to map the array, I need to reference it and instead, the UI is only showing me items from the array, not the array itself. The array here is nested inside the array that I need to map.

[Msquare_Automation]

@alexs

This is because you are not having these items into one array. Its in bundles. To get the main array that is having these netsed arrays, you will need to first aggregate all these bundles. Then you will be able to map.

If you require additional assistance, please don’t hesitate to reach out to us.
MSquare Support | Book Live Implementation
Visit us here
Youtube Channel

[alexs]

@Msquare_Automation

This is because you are not having these items into one array. Its in bundles. To get the main array that is having these netsed arrays, you will need to first aggregate all these bundles. Then you will be able to map.

That was my understanding too but the other Alex was saying that, that might not be necessary -

@alex.newpath

@alexs

Seems crazy that you have to add an array aggregator after an array aggregator to get the output into the correct format

I am not sure you have to do that actually – I was working with the JSON snippet for the structure you gave me and the Parse JSON module de-aggregates the aggregated array so that’s why I reaggregated it.

You can put the expressions right after the first aggregator.

[alex.newpath]

@alexs

CleanShot 2024-06-14 at 20.07.20@2x1744×1158 147 KB

Looking back to your original snapshot, it’s clear that the array aggregator is generating multiple bundles which is odd since your source module appears to be the Notion module which has 1 operation being used so I would think it shold output only 1 bundle with an array, so yes you’d need to aggregate again after the aggregator to get all output bundles into 1 bundle and 1 array.

[alexs]

Got it, thanks for confirming!