Make for Make Newbies Video Series I: All About that Data

I am undertaking a little exercise to explain the core features of Make using a scenario walkthrough and a related video. My hope is to record videos between 5-30 minutes long to explain Make to Make Newbies.

I am going to start with a topic which I think confuses just about everyone who starts using Make but has never programmed before:

Part 1: Exploring Data Structures for Make Newbies

In this video and accompanying scenario we will review simple arrays and more complex collections and arrays of collections. The interaction between the number of elements in an array and how bundles are created is also covered. Iterators are covered which can loop an array and generate processing loops in scenarios. The Parse JSON module is also introduced which is useful for parsing text which contains JSON into Make-friendly data structures. By the way JSON, is JavaScript Object Notation, and it is the “lingua-franca” of all data that is passed between modules of a Make scenario. It is also the structure of how most online apps expect data to sent via APIs. I highly recommend you get comfortable with JSON!

And here’s a tip for creating your own JSON: sample JSON can be created in ChatGPT just by describing the type of JSON you want in plain text. If you have some JSON you wrote and want to validate or pretty up the format, use JSONLint.

Copy and Paste this blueprint into a new Make Scenario
{
    "subflows": [
        {
            "flow": [
                {
                    "id": 3,
                    "module": "util:SetVariables",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "variables": [
                            {
                                "name": "Simple array",
                                "value": "{{add(emptyarray; \"apple\"; \"banana\"; \"cherry\"; 42; true; now; null)}}"
                            },
                            {
                                "name": "JSON simple collection",
                                "value": "{\n    \"name\": \"Alex C\",\n    \"age\": 2,\n    \"city\": \"Houston\",\n    \"date_entered\": \"{{now}}\"\n}"
                            },
                            {
                                "name": "JSON array of simple collections",
                                "value": "[\n    {\n        \"name\": \"Alex C\",\n        \"age\": 53,\n        \"city\": \"Toronto\"\n    },\n    {\n        \"name\": \"Eric S\",\n        \"age\": 43,\n        \"city\": \"Hoboken\"\n    }\n]"
                            },
                            {
                                "name": "JSON collection complex values",
                                "value": "{\n\t\"name\": \"Aleix Melon\",\n\t\"id\": \"E00245\",\n\t\"role\": [\"Dev\", \"DBA\"],\n\t\"age\": 23,\n\t\"doj\": \"11-12-2019\",\n\t\"married\": {{false}},\n\t\"address\": {\n\t\t\"street\": \"32, Laham St.\",\n\t\t\"city\": \"Innsbruck\",\n\t\t\"country\": \"Austria\"\n\t},\n\t\"referred-by\": \"E0012\"\n}"
                            },
                            {
                                "name": "JSON array of collections complex values",
                                "value": "[\n\t{\n        \"name\": \"Aleix Melon\",\n        \"id\": \"E00245\",\n        \"role\": [\"Dev\", \"DBA\"],\n        \"age\": 23,\n        \"doj\": \"11-12-2019\",\n        \"married\": {{false}},\n        \"address\": {\n            \"street\": \"32, Laham St.\",\n            \"city\": \"Innsbruck\",\n            \"country\": \"Austria\"\n            },\n        \"referred-by\": \"E0012\"\n\t},\n{\n      \"name\": \"Bob Washington\",\n        \"id\": \"E01245\",\n        \"role\": [\"HR\"],\n        \"age\": 43,\n        \"doj\": \"10-06-2010\",\n        \"married\": {{true}},\n        \"address\": {\n            \"street\": \"45, Abraham Lane.\",\n            \"city\": \"Washington\",\n            \"country\": \"USA\"\n            },\n        \"referred-by\": \"\"\n\t}\n]"
                            },
                            {
                                "name": "JSON array of collections of really complex values",
                                "value": "[\n    {\n        \"item_name\": \"Email campaign\",\n        \"item_sku_attachment\": \"\",\n        \"item_options\": [\n            {\n                \"Organization name\": \"Client A\"\n            },\n            {\n                \"Writing services\": \"Writing service A\"\n            },\n            {\n                \"Design services\": \"Design service B\"\n            },\n            {\n                \"Comments\": \"test comment\"\n            }\n        ],\n        \"item_image\": \"image.png\",\n        \"unit_price\": 0,\n        \"quantity\": 5,\n        \"weight\": 0,\n        \"created_at\": \"2024-05-11 13:37:46\",\n        \"product_price\": 195,\n        \"categories\": [\n            {\n                \"id\": \"cat_6633e1936571b618828100\",\n                \"category_name\": \"Send\",\n                \"category_slug\": \"send\",\n                \"status\": \"1\"\n            }\n        ]\n    },\n    {\n        \"item_name\": \"Ad campaign\",\n        \"item_sku_attachment\": \"\",\n        \"item_options\": [\n            {\n                \"Organization name\": \"Client B\"\n            },\n            {\n                \"Writing services\": \"Writing service B\"\n            },\n            {\n                \"Design services\": \"Design service C\"\n            },\n            {\n                \"Comments\": \"test comment 2\"\n            }\n        ],\n        \"item_image\": \"image.png\",\n        \"unit_price\": 0,\n        \"quantity\": 5,\n        \"weight\": 0,\n        \"created_at\": \"2023-05-11 13:37:46\",\n        \"product_price\": 1295,\n        \"categories\": [\n            {\n                \"id\": \"cat_6633e1936571b618828100\",\n                \"category_name\": \"Send\",\n                \"category_slug\": \"send\",\n                \"status\": \"1\"\n            }\n        ]\n    }\n]"
                            }
                        ],
                        "scope": "roundtrip"
                    },
                    "metadata": {
                        "designer": {
                            "x": 0,
                            "y": 150
                        },
                        "restore": {
                            "expect": {
                                "variables": {
                                    "items": [
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null
                                    ]
                                },
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "variables",
                                "type": "array",
                                "label": "Variables",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Variable name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Variable value",
                                        "type": "any"
                                    }
                                ]
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            }
                        ],
                        "interface": [
                            {
                                "name": "Simple array",
                                "label": "Simple array",
                                "type": "any"
                            },
                            {
                                "name": "JSON simple collection",
                                "label": "JSON simple collection",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of simple collections",
                                "label": "JSON array of simple collections",
                                "type": "any"
                            },
                            {
                                "name": "JSON collection complex values",
                                "label": "JSON collection complex values",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of collections complex values",
                                "label": "JSON array of collections complex values",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of collections of really complex values",
                                "label": "JSON array of collections of really complex values",
                                "type": "any"
                            }
                        ]
                    }
                },
                {
                    "id": 5,
                    "module": "builtin:BasicFeeder",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "array": [
                            "{{3.`Simple array`}}",
                            "{{3.`JSON simple collection`}}",
                            "{{3.`JSON array of simple collections`}}",
                            "{{3.`JSON collection complex values`}}",
                            "{{3.`JSON array of collections complex values`}}",
                            "{{3.`JSON array of collections of really complex values`}}"
                        ]
                    },
                    "metadata": {
                        "designer": {
                            "x": 300,
                            "y": 150
                        },
                        "restore": {
                            "expect": {
                                "array": {
                                    "mode": "chose",
                                    "items": [
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null
                                    ]
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "array",
                                "type": "array",
                                "label": "Array",
                                "mode": "edit",
                                "spec": []
                            }
                        ]
                    }
                },
                {
                    "id": 6,
                    "module": "builtin:BasicRouter",
                    "version": 1,
                    "parameters": {
                        "else": 1
                    },
                    "mapper": null,
                    "metadata": {
                        "designer": {
                            "x": 600,
                            "y": 150
                        }
                    },
                    "routes": [
                        {
                            "flow": [
                                {
                                    "id": 2,
                                    "module": "builtin:BasicFeeder",
                                    "version": 1,
                                    "parameters": {},
                                    "filter": {
                                        "name": "First bundle",
                                        "conditions": [
                                            [
                                                {
                                                    "a": "{{5.`__IMTINDEX__`}}",
                                                    "o": "number:equal",
                                                    "b": "1"
                                                }
                                            ]
                                        ]
                                    },
                                    "mapper": {
                                        "array": "{{5.value}}"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 900,
                                            "y": 0,
                                            "messages": [
                                                {
                                                    "category": "last",
                                                    "severity": "warning",
                                                    "message": "A transformer should not be the last module in the route."
                                                }
                                            ]
                                        },
                                        "restore": {
                                            "expect": {
                                                "array": {
                                                    "mode": "edit",
                                                    "items": [
                                                        null,
                                                        null,
                                                        null,
                                                        null
                                                    ]
                                                }
                                            }
                                        },
                                        "expect": [
                                            {
                                                "name": "array",
                                                "type": "array",
                                                "label": "Array",
                                                "mode": "edit",
                                                "spec": []
                                            }
                                        ]
                                    }
                                }
                            ]
                        },
                        {
                            "flow": [
                                {
                                    "id": 7,
                                    "module": "json:ParseJSON",
                                    "version": 1,
                                    "parameters": {
                                        "type": ""
                                    },
                                    "mapper": {
                                        "json": "{{5.value}}"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 900,
                                            "y": 300,
                                            "messages": [
                                                {
                                                    "category": "last",
                                                    "severity": "warning",
                                                    "message": "A transformer should not be the last module in the route."
                                                }
                                            ]
                                        },
                                        "restore": {
                                            "parameters": {
                                                "type": {
                                                    "label": "Choose a data structure"
                                                }
                                            }
                                        },
                                        "parameters": [
                                            {
                                                "name": "type",
                                                "type": "udt",
                                                "label": "Data structure"
                                            }
                                        ],
                                        "expect": [
                                            {
                                                "name": "json",
                                                "type": "text",
                                                "label": "JSON string",
                                                "required": true
                                            }
                                        ]
                                    }
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

7 Likes

How exciting @alex.newpath! :clap: :clap: :clap:

This is a fantastic idea and an incredibly valuable initiative. Thank you so much for thinking of our Make newbies and for helping them explore the amazing possibilities Make offers! :star2:

1 Like

Of course! The more Makers are successful with their projects the more automation they will attempt and hopefully succeed with. Refining your Make skills is like any other skill like playing a sport or a musical instrument. You gotta practice!

3 Likes

Here’s Part 2: Exploring Aggregators for Make Newbies

In this video we introduce a new complex JSON structure that describes mutiple library systems that can be used in future sessions related to array functions. We also compare Array Aggregators with array functions like merge(), join() and remove(). We apply these functions to our simplest array so you can get more familiar. This video is pretty short at just under 10 minutes long.

Copy and Paste this blueprint into a new Make Scenario
{
    "subflows": [
        {
            "flow": [
                {
                    "id": 3,
                    "module": "util:SetVariables",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "variables": [
                            {
                                "name": "Simple array",
                                "value": "{{add(emptyarray; \"apple\"; \"banana\"; \"cherry\"; 42; true; null; now)}}"
                            },
                            {
                                "name": "another simple array",
                                "value": "{{add(emptyarray; \"pear\"; \"mango\"; 123; false)}}"
                            },
                            {
                                "name": "JSON simple collection",
                                "value": "{\n    \"name\": \"Alex C\",\n    \"age\": 2,\n    \"city\": \"Houston\",\n    \"date_entered\": \"{{now}}\"\n}"
                            },
                            {
                                "name": "JSON array of simple collections",
                                "value": "[\n    {\n        \"name\": \"Alex C\",\n        \"age\": 53,\n        \"city\": \"Toronto\"\n    },\n    {\n        \"name\": \"Eric S\",\n        \"age\": 43,\n        \"city\": \"Hoboken\"\n    }\n]"
                            },
                            {
                                "name": "JSON collection complex values",
                                "value": "{\n\t\"name\": \"Aleix Melon\",\n\t\"id\": \"E00245\",\n\t\"role\": [\"Dev\", \"DBA\"],\n\t\"age\": 23,\n\t\"doj\": \"11-12-2019\",\n\t\"married\": {{false}},\n\t\"address\": {\n\t\t\"street\": \"32, Laham St.\",\n\t\t\"city\": \"Innsbruck\",\n\t\t\"country\": \"Austria\"\n\t},\n\t\"referred-by\": \"E0012\"\n}"
                            },
                            {
                                "name": "JSON array of collections complex values",
                                "value": "[\n\t{\n        \"name\": \"Aleix Melon\",\n        \"id\": \"E00245\",\n        \"role\": [\"Dev\", \"DBA\"],\n        \"age\": 23,\n        \"doj\": \"11-12-2019\",\n        \"married\": {{false}},\n        \"address\": {\n            \"street\": \"32, Laham St.\",\n            \"city\": \"Innsbruck\",\n            \"country\": \"Austria\"\n            },\n        \"referred-by\": \"E0012\"\n\t},\n{\n      \"name\": \"Bob Washington\",\n        \"id\": \"E01245\",\n        \"role\": [\"HR\"],\n        \"age\": 43,\n        \"doj\": \"10-06-2010\",\n        \"married\": {{true}},\n        \"address\": {\n            \"street\": \"45, Abraham Lane.\",\n            \"city\": \"Washington\",\n            \"country\": \"USA\"\n            },\n        \"referred-by\": \"\"\n\t}\n]"
                            },
                            {
                                "name": "JSON array of collections of really complex values",
                                "value": "[\n    {\n        \"item_name\": \"Email campaign\",\n        \"item_sku_attachment\": \"\",\n        \"item_options\": [\n            {\n                \"Organization name\": \"Client A\"\n            },\n            {\n                \"Writing services\": \"Writing service A\"\n            },\n            {\n                \"Design services\": \"Design service B\"\n            },\n            {\n                \"Comments\": \"test comment\"\n            }\n        ],\n        \"item_image\": \"image.png\",\n        \"unit_price\": 0,\n        \"quantity\": 5,\n        \"weight\": 0,\n        \"created_at\": \"2024-05-11 13:37:46\",\n        \"product_price\": 195,\n        \"categories\": [\n            {\n                \"id\": \"cat_6633e1936571b618828100\",\n                \"category_name\": \"Send\",\n                \"category_slug\": \"send\",\n                \"status\": \"1\"\n            }\n        ]\n    },\n    {\n        \"item_name\": \"Ad campaign\",\n        \"item_sku_attachment\": \"\",\n        \"item_options\": [\n            {\n                \"Organization name\": \"Client B\"\n            },\n            {\n                \"Writing services\": \"Writing service B\"\n            },\n            {\n                \"Design services\": \"Design service C\"\n            },\n            {\n                \"Comments\": \"test comment 2\"\n            }\n        ],\n        \"item_image\": \"image.png\",\n        \"unit_price\": 0,\n        \"quantity\": 5,\n        \"weight\": 0,\n        \"created_at\": \"2023-05-11 13:37:46\",\n        \"product_price\": 1295,\n        \"categories\": [\n            {\n                \"id\": \"cat_6633e1936571b618828100\",\n                \"category_name\": \"Send\",\n                \"category_slug\": \"send\",\n                \"status\": \"1\"\n            }\n        ]\n    }\n]"
                            },
                            {
                                "name": "JSON Library Systems",
                                "value": "[{\n  \"library\": {\n    \"name\": \"Central Library\",\n    \"location\": \"123 Main Street\",\n    \"catalog\": {\n      \"sections\": [\n        {\n          \"name\": \"Fiction\",\n          \"books\": [\n            {\n              \"title\": \"To Kill a Mockingbird\",\n              \"author\": \"Harper Lee\",\n              \"genre\": \"Classic\",\n              \"copies\": 5\n            },\n            {\n              \"title\": \"1984\",\n              \"author\": \"George Orwell\",\n              \"genre\": \"Dystopian\",\n              \"copies\": 3\n            }\n          ]\n        },\n        {\n          \"name\": \"Non-Fiction\",\n          \"books\": [\n            {\n              \"title\": \"Sapiens: A Brief History of Humankind\",\n              \"author\": \"Yuval Noah Harari\",\n              \"genre\": \"History\",\n              \"copies\": 7\n            },\n            {\n              \"title\": \"The Power of Habit\",\n              \"author\": \"Charles Duhigg\",\n              \"genre\": \"Psychology\",\n              \"copies\": 4\n            }\n          ]\n        }\n      ]\n    }\n  }\n},\n{\n  \"library\": {\n    \"name\": \"Downtown Library\",\n    \"location\": \"456 Elm Street\",\n    \"catalog\": {\n      \"sections\": [\n        {\n          \"name\": \"Science Fiction\",\n          \"books\": [\n            {\n              \"title\": \"Dune\",\n              \"author\": \"Frank Herbert\",\n              \"genre\": \"Science Fiction\",\n              \"copies\": 6\n            },\n            {\n              \"title\": \"Neuromancer\",\n              \"author\": \"William Gibson\",\n              \"genre\": \"Cyberpunk\",\n              \"copies\": 4\n            }\n          ]\n        },\n        {\n          \"name\": \"Biography\",\n          \"books\": [\n            {\n              \"title\": \"Steve Jobs\",\n              \"author\": \"Walter Isaacson\",\n              \"genre\": \"Biography\",\n              \"copies\": 3\n            },\n            {\n              \"title\": \"The Diary of a Young Girl\",\n              \"author\": \"Anne Frank\",\n              \"genre\": \"Autobiography\",\n              \"copies\": 5\n            }\n          ]\n        }\n      ]\n    }\n  }\n},\n{\n  \"library\": {\n    \"name\": \"Westside Library\",\n    \"location\": \"789 Oak Avenue\",\n    \"catalog\": {\n      \"sections\": [\n        {\n          \"name\": \"Mystery\",\n          \"books\": [\n            {\n              \"title\": \"The Girl with the Dragon Tattoo\",\n              \"author\": \"Stieg Larsson\",\n              \"genre\": \"Mystery\",\n              \"copies\": 8\n            },\n            {\n              \"title\": \"Gone Girl\",\n              \"author\": \"Gillian Flynn\",\n              \"genre\": \"Thriller\",\n              \"copies\": 6\n            }\n          ]\n        },\n        {\n          \"name\": \"History\",\n          \"books\": [\n            {\n              \"title\": \"The Guns of August\",\n              \"author\": \"Barbara W. Tuchman\",\n              \"genre\": \"History\",\n              \"copies\": 4\n            },\n            {\n              \"title\": \"1491\",\n              \"author\": \"Charles C. Mann\",\n              \"genre\": \"Archaeology\",\n              \"copies\": 7\n            }\n          ]\n        }\n      ]\n    }\n  }\n},\n{\n  \"library\": {\n    \"name\": \"Eastside Library\",\n    \"location\": \"1010 Pine Boulevard\",\n    \"catalog\": {\n      \"sections\": [\n        {\n          \"name\": \"Fantasy\",\n          \"books\": [\n            {\n              \"title\": \"Harry Potter and the Philosopher's Stone\",\n              \"author\": \"J.K. Rowling\",\n              \"genre\": \"Fantasy\",\n              \"copies\": 10\n            },\n            {\n              \"title\": \"The Hobbit\",\n              \"author\": \"J.R.R. Tolkien\",\n              \"genre\": \"Adventure\",\n              \"copies\": 8\n            }\n          ]\n        },\n        {\n          \"name\": \"Self-Help\",\n          \"books\": [\n            {\n              \"title\": \"The Subtle Art of Not Giving a F*ck\",\n              \"author\": \"Mark Manson\",\n              \"genre\": \"Self-Help\",\n              \"copies\": 5\n            },\n            {\n              \"title\": \"Atomic Habits\",\n              \"author\": \"James Clear\",\n              \"genre\": \"Personal Development\",\n              \"copies\": 6\n            }\n          ]\n        }\n      ]\n    }\n  }\n},\n{\n  \"library\": {\n    \"name\": \"Southside Library\",\n    \"location\": \"1212 Maple Street\",\n    \"catalog\": {\n      \"sections\": [\n        {\n          \"name\": \"Fantasy\",\n          \"books\": [\n            {\n              \"title\": \"The Name of the Wind\",\n              \"author\": \"Patrick Rothfuss\",\n              \"genre\": \"Fantasy\",\n              \"copies\": 7\n            },\n            {\n              \"title\": \"Mistborn: The Final Empire\",\n              \"author\": \"Brandon Sanderson\",\n              \"genre\": \"Fantasy\",\n              \"copies\": 6\n            }\n          ]\n        },\n        {\n          \"name\": \"Self-Help\",\n          \"books\": [\n            {\n              \"title\": \"The 7 Habits of Highly Effective People\",\n              \"author\": \"Stephen R. Covey\",\n              \"genre\": \"Self-Help\",\n              \"copies\": 8\n            },\n            {\n              \"title\": \"How to Win Friends and Influence People\",\n              \"author\": \"Dale Carnegie\",\n              \"genre\": \"Self-Help\",\n              \"copies\": 9\n            }\n          ]\n        }\n      ]\n    }\n  }\n}\n]"
                            }
                        ],
                        "scope": "roundtrip"
                    },
                    "metadata": {
                        "designer": {
                            "x": 0,
                            "y": 150
                        },
                        "restore": {
                            "expect": {
                                "variables": {
                                    "items": [
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null
                                    ]
                                },
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "variables",
                                "type": "array",
                                "label": "Variables",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Variable name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Variable value",
                                        "type": "any"
                                    }
                                ]
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            }
                        ],
                        "interface": [
                            {
                                "name": "Simple array",
                                "label": "Simple array",
                                "type": "any"
                            },
                            {
                                "name": "another simple array",
                                "label": "another simple array",
                                "type": "any"
                            },
                            {
                                "name": "JSON simple collection",
                                "label": "JSON simple collection",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of simple collections",
                                "label": "JSON array of simple collections",
                                "type": "any"
                            },
                            {
                                "name": "JSON collection complex values",
                                "label": "JSON collection complex values",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of collections complex values",
                                "label": "JSON array of collections complex values",
                                "type": "any"
                            },
                            {
                                "name": "JSON array of collections of really complex values",
                                "label": "JSON array of collections of really complex values",
                                "type": "any"
                            },
                            {
                                "name": "JSON Library Systems",
                                "label": "JSON Library Systems",
                                "type": "any"
                            }
                        ]
                    }
                },
                {
                    "id": 5,
                    "module": "builtin:BasicFeeder",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "array": [
                            "{{3.`Simple array`}}",
                            "{{3.`another simple array`}}",
                            "{{3.`JSON simple collection`}}",
                            "{{3.`JSON array of simple collections`}}",
                            "{{3.`JSON collection complex values`}}",
                            "{{3.`JSON array of collections complex values`}}",
                            "{{3.`JSON array of collections of really complex values`}}",
                            "{{3.`JSON Library Systems`}}"
                        ]
                    },
                    "metadata": {
                        "designer": {
                            "x": 300,
                            "y": 150
                        },
                        "restore": {
                            "expect": {
                                "array": {
                                    "mode": "chose",
                                    "items": [
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null
                                    ]
                                }
                            }
                        },
                        "expect": [
                            {
                                "mode": "edit",
                                "name": "array",
                                "spec": [],
                                "type": "array",
                                "label": "Array"
                            }
                        ]
                    }
                },
                {
                    "id": 6,
                    "module": "builtin:BasicRouter",
                    "version": 1,
                    "parameters": {
                        "else": 1
                    },
                    "mapper": null,
                    "metadata": {
                        "designer": {
                            "x": 600,
                            "y": 150
                        }
                    },
                    "routes": [
                        {
                            "flow": [
                                {
                                    "id": 2,
                                    "module": "builtin:BasicFeeder",
                                    "version": 1,
                                    "parameters": {},
                                    "filter": {
                                        "name": "First or second bundle",
                                        "conditions": [
                                            [
                                                {
                                                    "a": "{{5.`__IMTINDEX__`}}",
                                                    "b": "1",
                                                    "o": "number:equal"
                                                }
                                            ],
                                            [
                                                {
                                                    "a": "{{5.`__IMTINDEX__`}}",
                                                    "b": "2",
                                                    "o": "number:equal"
                                                }
                                            ]
                                        ]
                                    },
                                    "mapper": {
                                        "array": "{{5.value}}"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 900,
                                            "y": 0
                                        },
                                        "restore": {
                                            "expect": {
                                                "array": {
                                                    "mode": "edit",
                                                    "items": [
                                                        null,
                                                        null,
                                                        null,
                                                        null
                                                    ]
                                                }
                                            }
                                        },
                                        "expect": [
                                            {
                                                "mode": "edit",
                                                "name": "array",
                                                "spec": [],
                                                "type": "array",
                                                "label": "Array"
                                            }
                                        ]
                                    }
                                },
                                {
                                    "id": 9,
                                    "module": "builtin:BasicAggregator",
                                    "version": 1,
                                    "parameters": {
                                        "feeder": 5
                                    },
                                    "mapper": {
                                        "value": "{{2.value}}"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 1200,
                                            "y": 0
                                        },
                                        "restore": {
                                            "extra": {
                                                "feeder": {
                                                    "label": "Iterator [5]"
                                                },
                                                "target": {
                                                    "label": "Custom"
                                                }
                                            }
                                        }
                                    }
                                },
                                {
                                    "id": 10,
                                    "module": "util:SetVariables",
                                    "version": 1,
                                    "parameters": {},
                                    "mapper": {
                                        "variables": [
                                            {
                                                "name": "merged array",
                                                "value": "{{merge(3.`Simple array`; 3.`another simple array`)}}"
                                            },
                                            {
                                                "name": "Array as a string with delimiters",
                                                "value": "{{join(remove(3.`Simple array`; null); \",\" + space)}}"
                                            }
                                        ],
                                        "scope": "roundtrip"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 1500,
                                            "y": 0
                                        },
                                        "restore": {
                                            "expect": {
                                                "variables": {
                                                    "items": [
                                                        null,
                                                        null
                                                    ]
                                                },
                                                "scope": {
                                                    "label": "One cycle"
                                                }
                                            }
                                        },
                                        "expect": [
                                            {
                                                "name": "variables",
                                                "type": "array",
                                                "label": "Variables",
                                                "spec": [
                                                    {
                                                        "name": "name",
                                                        "label": "Variable name",
                                                        "type": "text",
                                                        "required": true
                                                    },
                                                    {
                                                        "name": "value",
                                                        "label": "Variable value",
                                                        "type": "any"
                                                    }
                                                ]
                                            },
                                            {
                                                "name": "scope",
                                                "type": "select",
                                                "label": "Variable lifetime",
                                                "required": true,
                                                "validate": {
                                                    "enum": [
                                                        "roundtrip",
                                                        "execution"
                                                    ]
                                                }
                                            }
                                        ],
                                        "interface": [
                                            {
                                                "name": "merged array",
                                                "label": "merged array",
                                                "type": "any"
                                            },
                                            {
                                                "name": "Array as a string with delimiters",
                                                "label": "Array as a string with delimiters",
                                                "type": "any"
                                            }
                                        ]
                                    }
                                }
                            ]
                        },
                        {
                            "flow": [
                                {
                                    "id": 7,
                                    "module": "json:ParseJSON",
                                    "version": 1,
                                    "parameters": {
                                        "type": ""
                                    },
                                    "mapper": {
                                        "json": "{{5.value}}"
                                    },
                                    "metadata": {
                                        "designer": {
                                            "x": 900,
                                            "y": 300,
                                            "messages": [
                                                {
                                                    "category": "last",
                                                    "severity": "warning",
                                                    "message": "A transformer should not be the last module in the route."
                                                }
                                            ]
                                        },
                                        "restore": {
                                            "parameters": {
                                                "type": {
                                                    "label": "Choose a data structure"
                                                }
                                            }
                                        },
                                        "parameters": [
                                            {
                                                "name": "type",
                                                "type": "udt",
                                                "label": "Data structure"
                                            }
                                        ],
                                        "expect": [
                                            {
                                                "name": "json",
                                                "type": "text",
                                                "label": "JSON string",
                                                "required": true
                                            }
                                        ]
                                    }
                                }
                            ]
                        }
                    ]
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

Here’s Part 3: Exploring Array Functions and Text Aggregators for Make Newbies - Part A

In this video we describe sevearl useful array functions like first() and last(). We also look at how to reference a particular element of an array by index using 2 methods - using dot notation and with the “star” UI element picker. We also look at how to reference values from a collection coming out of an array aggregator. This Make help file is of relevance.

Download the Make blueprint and use the Import Blueprint to create a new scenario:

3. Exploring Array Functions and Text Aggregators for Make Newbies.json (39.8 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

Here’s Part 3: Exploring Array Functions and Text Aggregators for Make Newbies - Part B

In this video I continue to explore our variables and combine an array of one set of key value pairs, and another array of 2 elements with key value pairs using text aggregators.

Download the Make blueprint and use the Import Blueprint to create a new scenario:

3. Exploring Array Functions and Text Aggregators for Make Newbies.json (39.8 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

Here’s a little background on the text variables I set in the first module of our sample scenario. You may like to see the different types of structures to study them to understand their differences. Make expressions can be copied and pasted into Make input boxes in most places.


Variable name: Simple array

Make expression: {{add(emptyarray; "apple"; "banana"; "cherry"; 42; true; null; now)}}

Bundle output
 {
        "Simple array": [
            "apple",
            "banana",
            "cherry",
            42,
            true,
            null,
            "2024-05-23T20:07:14.697Z"
        ]
  }

Variable name: another simple array

Make expression: {{add(emptyarray; "pear"; "mango"; 123; false)}}

Bundle output
 {
      "another simple array": [
            "pear",
            "mango",
            123,
            false
        ]
  }

Variable name: JSON simple collection

Make expression: { "name": "Alex C", "age": 2, "city": "Houston", "date_entered": "{{now}}" }

Bundle output
    {
    "name": "Alex C",
    "age": 2,
    "city": "Houston",
    "date_entered": "2024-05-23T20:07:14.698Z"
    }

Variable name: JSON array of simple collections

Make expression: [ { "name": "Alex C", "age": 53, "city": "Toronto" }, { "name": "Eric S", "age": 43, "city": "Hoboken" } ]

Bundle output
[
    {
        "name": "Alex C",
        "age": 53,
        "city": "Toronto"
    },
    {
        "name": "Eric S",
        "age": 43,
        "city": "Hoboken"
    }
]

Variable name: JSON collection complex values

Make expression: { "name": "Aleix Melon", "id": "E00245", "role": ["Dev", "DBA"], "age": 23, "doj": "11-12-2019", "married": {{false}}, "address": { "street": "32, Laham St.", "city": "Innsbruck", "country": "Austria" }, "referred-by": "E0012" }

Bundle output
{
    "name": "Aleix Melon",
    "id": "E00245",
    "role": [
        "Dev",
        "DBA"
    ],
    "age": 23,
    "doj": "11-12-2019",
    "married": false,
    "address": {
        "street": "32, Laham St.",
        "city": "Innsbruck",
        "country": "Austria"
    },
    "referred-by": "E0012"
}

Variable name: JSON array of collections complex values

Make expression: [ { "name": "Aleix Melon", "id": "E00245", "role": ["Dev", "DBA"], "age": 23, "doj": "11-12-2019", "married": {{false}}, "address": { "street": "32, Laham St.", "city": "Innsbruck", "country": "Austria" }, "referred-by": "E0012" }, { "name": "Bob Washington", "id": "E01245", "role": ["HR"], "age": 43, "doj": "10-06-2010", "married": {{true}}, "address": { "street": "45, Abraham Lane.", "city": "Washington", "country": "USA" }, "referred-by": "" } ]

Bundle output
[
    {
        "name": "Aleix Melon",
        "id": "E00245",
        "role": [
            "Dev",
            "DBA"
        ],
        "age": 23,
        "doj": "11-12-2019",
        "married": false,
        "address": {
            "street": "32, Laham St.",
            "city": "Innsbruck",
            "country": "Austria"
        },
        "referred-by": "E0012"
    },
    {
        "name": "Bob Washington",
        "id": "E01245",
        "role": [
            "HR"
        ],
        "age": 43,
        "doj": "10-06-2010",
        "married": true,
        "address": {
            "street": "45, Abraham Lane.",
            "city": "Washington",
            "country": "USA"
        },
        "referred-by": ""
    }
]

Variable name: JSON array of collections of really complex values

Make expression: [ { "item_name": "Email campaign", "item_sku_attachment": "", "item_options": [ { "Organization name": "Client A" }, { "Writing services": "Writing service A" }, { "Design services": "Design service B" }, { "Comments": "test comment" } ], "item_image": "image.png", "unit_price": 0, "quantity": 5, "weight": 0, "created_at": "2024-05-11 13:37:46", "product_price": 195, "categories": [ { "id": "cat_6633e1936571b618828100", "category_name": "Send", "category_slug": "send", "status": "1" } ] }, { "item_name": "Ad campaign", "item_sku_attachment": "", "item_options": [ { "Organization name": "Client B" }, { "Writing services": "Writing service B" }, { "Design services": "Design service C" }, { "Comments": "test comment 2" } ], "item_image": "image.png", "unit_price": 0, "quantity": 5, "weight": 0, "created_at": "2023-05-11 13:37:46", "product_price": 1295, "categories": [ { "id": "cat_6633e1936571b618828100", "category_name": "Send", "category_slug": "send", "status": "1" } ] } ]

Bundle output
[
    {
        "item_name": "Email campaign",
        "item_sku_attachment": "",
        "item_options": [
            {
                "Organization name": "Client A"
            },
            {
                "Writing services": "Writing service A"
            },
            {
                "Design services": "Design service B"
            },
            {
                "Comments": "test comment"
            }
        ],
        "item_image": "image.png",
        "unit_price": 0,
        "quantity": 5,
        "weight": 0,
        "created_at": "2024-05-11 13:37:46",
        "product_price": 195,
        "categories": [
            {
                "id": "cat_6633e1936571b618828100",
                "category_name": "Send",
                "category_slug": "send",
                "status": "1"
            }
        ]
    },
    {
        "item_name": "Ad campaign",
        "item_sku_attachment": "",
        "item_options": [
            {
                "Organization name": "Client B"
            },
            {
                "Writing services": "Writing service B"
            },
            {
                "Design services": "Design service C"
            },
            {
                "Comments": "test comment 2"
            }
        ],
        "item_image": "image.png",
        "unit_price": 0,
        "quantity": 5,
        "weight": 0,
        "created_at": "2023-05-11 13:37:46",
        "product_price": 1295,
        "categories": [
            {
                "id": "cat_6633e1936571b618828100",
                "category_name": "Send",
                "category_slug": "send",
                "status": "1"
            }
        ]
    }
]

Variable name: JSON Library Systems

Make expression:
[{ "library": { "libraryname": "Central Library", "location": "123 Main Street", "catalog": { "sections": [ { "name": "Fiction", "books": [ { "title": "To Kill a Mockingbird", "author": "Harper Lee", "genre": "Classic", "copies": 5 }, { "title": "1984", "author": "George Orwell", "genre": "Dystopian", "copies": 3 } ] }, { "name": "Non-Fiction", "books": [ { "title": "Sapiens: A Brief History of Humankind", "author": "Yuval Noah Harari", "genre": "History", "copies": 7 }, { "title": "The Power of Habit", "author": "Charles Duhigg", "genre": "Psychology", "copies": 4 } ] } ] } } }, { "library": { "libraryname": "Downtown Library", "location": "456 Elm Street", "catalog": { "sections": [ { "name": "Science Fiction", "books": [ { "title": "Dune", "author": "Frank Herbert", "genre": "Science Fiction", "copies": 6 }, { "title": "Neuromancer", "author": "William Gibson", "genre": "Cyberpunk", "copies": 4 } ] }, { "name": "Biography", "books": [ { "title": "Steve Jobs", "author": "Walter Isaacson", "genre": "Biography", "copies": 3 }, { "title": "The Diary of a Young Girl", "author": "Anne Frank", "genre": "Autobiography", "copies": 5 } ] } ] } } }, { "library": { "libraryname": "Westside Library", "location": "789 Oak Avenue", "catalog": { "sections": [ { "name": "Mystery", "books": [ { "title": "The Girl with the Dragon Tattoo", "author": "Stieg Larsson", "genre": "Mystery", "copies": 8 }, { "title": "Gone Girl", "author": "Gillian Flynn", "genre": "Thriller", "copies": 6 } ] }, { "name": "History", "books": [ { "title": "The Guns of August", "author": "Barbara W. Tuchman", "genre": "History", "copies": 4 }, { "title": "1491", "author": "Charles C. Mann", "genre": "Archaeology", "copies": 7 } ] } ] } } }, { "library": { "libraryname": "Eastside Library", "location": "1010 Pine Boulevard", "catalog": { "sections": [ { "name": "Fantasy", "books": [ { "title": "Harry Potter and the Philosopher's Stone", "author": "J.K. Rowling", "genre": "Fantasy", "copies": 10 }, { "title": "The Hobbit", "author": "J.R.R. Tolkien", "genre": "Adventure", "copies": 8 } ] }, { "name": "Self-Help", "books": [ { "title": "The Subtle Art of Not Giving a F*ck", "author": "Mark Manson", "genre": "Self-Help", "copies": 5 }, { "title": "Atomic Habits", "author": "James Clear", "genre": "Personal Development", "copies": 6 } ] } ] } } }, { "library": { "libraryname": "Southside Library", "location": "1212 Maple Street", "catalog": { "sections": [ { "name": "Fantasy", "books": [ { "title": "The Name of the Wind", "author": "Patrick Rothfuss", "genre": "Fantasy", "copies": 7 }, { "title": "Mistborn: The Final Empire", "author": "Brandon Sanderson", "genre": "Fantasy", "copies": 6 } ] }, { "name": "Science Fiction", "books": [ { "title": "Do Androids Dream of Electric Sheep?", "author": "Philipp K. Dick", "genre": "Science Fiction", "copies": 1 } ] }, { "name": "Self-Help", "books": [ { "title": "The 7 Habits of Highly Effective People", "author": "Stephen R. Covey", "genre": "Self-Help", "copies": 8 }, { "title": "How to Win Friends and Influence People", "author": "Dale Carnegie", "genre": "Self-Help", "copies": 9 } ] } ] } } } ]

Bundle output
[
    {
        "library": {
            "libraryname": "Central Library",
            "location": "123 Main Street",
            "catalog": {
                "sections": [
                    {
                        "name": "Fiction",
                        "books": [
                            {
                                "title": "To Kill a Mockingbird",
                                "author": "Harper Lee",
                                "genre": "Classic",
                                "copies": 5
                            },
                            {
                                "title": "1984",
                                "author": "George Orwell",
                                "genre": "Dystopian",
                                "copies": 3
                            }
                        ]
                    },
                    {
                        "name": "Non-Fiction",
                        "books": [
                            {
                                "title": "Sapiens: A Brief History of Humankind",
                                "author": "Yuval Noah Harari",
                                "genre": "History",
                                "copies": 7
                            },
                            {
                                "title": "The Power of Habit",
                                "author": "Charles Duhigg",
                                "genre": "Psychology",
                                "copies": 4
                            }
                        ]
                    }
                ]
            }
        }
    },
    {
        "library": {
            "libraryname": "Downtown Library",
            "location": "456 Elm Street",
            "catalog": {
                "sections": [
                    {
                        "name": "Science Fiction",
                        "books": [
                            {
                                "title": "Dune",
                                "author": "Frank Herbert",
                                "genre": "Science Fiction",
                                "copies": 6
                            },
                            {
                                "title": "Neuromancer",
                                "author": "William Gibson",
                                "genre": "Cyberpunk",
                                "copies": 4
                            }
                        ]
                    },
                    {
                        "name": "Biography",
                        "books": [
                            {
                                "title": "Steve Jobs",
                                "author": "Walter Isaacson",
                                "genre": "Biography",
                                "copies": 3
                            },
                            {
                                "title": "The Diary of a Young Girl",
                                "author": "Anne Frank",
                                "genre": "Autobiography",
                                "copies": 5
                            }
                        ]
                    }
                ]
            }
        }
    },
    {
        "library": {
            "libraryname": "Westside Library",
            "location": "789 Oak Avenue",
            "catalog": {
                "sections": [
                    {
                        "name": "Mystery",
                        "books": [
                            {
                                "title": "The Girl with the Dragon Tattoo",
                                "author": "Stieg Larsson",
                                "genre": "Mystery",
                                "copies": 8
                            },
                            {
                                "title": "Gone Girl",
                                "author": "Gillian Flynn",
                                "genre": "Thriller",
                                "copies": 6
                            }
                        ]
                    },
                    {
                        "name": "History",
                        "books": [
                            {
                                "title": "The Guns of August",
                                "author": "Barbara W. Tuchman",
                                "genre": "History",
                                "copies": 4
                            },
                            {
                                "title": "1491",
                                "author": "Charles C. Mann",
                                "genre": "Archaeology",
                                "copies": 7
                            }
                        ]
                    }
                ]
            }
        }
    },
    {
        "library": {
            "libraryname": "Eastside Library",
            "location": "1010 Pine Boulevard",
            "catalog": {
                "sections": [
                    {
                        "name": "Fantasy",
                        "books": [
                            {
                                "title": "Harry Potter and the Philosopher's Stone",
                                "author": "J.K. Rowling",
                                "genre": "Fantasy",
                                "copies": 10
                            },
                            {
                                "title": "The Hobbit",
                                "author": "J.R.R. Tolkien",
                                "genre": "Adventure",
                                "copies": 8
                            }
                        ]
                    },
                    {
                        "name": "Self-Help",
                        "books": [
                            {
                                "title": "The Subtle Art of Not Giving a F*ck",
                                "author": "Mark Manson",
                                "genre": "Self-Help",
                                "copies": 5
                            },
                            {
                                "title": "Atomic Habits",
                                "author": "James Clear",
                                "genre": "Personal Development",
                                "copies": 6
                            }
                        ]
                    }
                ]
            }
        }
    },
    {
        "library": {
            "libraryname": "Southside Library",
            "location": "1212 Maple Street",
            "catalog": {
                "sections": [
                    {
                        "name": "Fantasy",
                        "books": [
                            {
                                "title": "The Name of the Wind",
                                "author": "Patrick Rothfuss",
                                "genre": "Fantasy",
                                "copies": 7
                            },
                            {
                                "title": "Mistborn: The Final Empire",
                                "author": "Brandon Sanderson",
                                "genre": "Fantasy",
                                "copies": 6
                            }
                        ]
                    },
                    {
                        "name": "Science Fiction",
                        "books": [
                            {
                                "title": "Do Androids Dream of Electric Sheep?",
                                "author": "Philipp K. Dick",
                                "genre": "Science Fiction",
                                "copies": 1
                            }
                        ]
                    },
                    {
                        "name": "Self-Help",
                        "books": [
                            {
                                "title": "The 7 Habits of Highly Effective People",
                                "author": "Stephen R. Covey",
                                "genre": "Self-Help",
                                "copies": 8
                            },
                            {
                                "title": "How to Win Friends and Influence People",
                                "author": "Dale Carnegie",
                                "genre": "Self-Help",
                                "copies": 9
                            }
                        ]
                    }
                ]
            }
        }
    }
]


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

2 Likes

Part 4: Managing More Complex Data Structures

In this walk through, we focus on the variable JSON collection complex values:

{
    "name": "Aleix Melon",
    "id": "E00245",
    "role": [
        "Dev",
        "DBA"
    ],
    "age": 23,
    "doj": "11-12-2019",
    "married": false,
    "address": {
        "street": "32, Laham St.",
        "city": "Innsbruck",
        "country": "Austria"
    },
    "referred-by": "E0012"
}

You’ll note that the key role is a simple array, and the key address is a collection composed of 3 keys: street, city, and country. This is why I call it a collection of “complex” values. The keys in this collection have nested non-trivial structures, and this happens A LOT in the data you have to deal with in APIs. In fact it’s pretty much a given that the structures you will deal with will be at least as complex as this. And note there is just 1 main collection here (READ: just 1 “row” of data is retrieved and some of the keys (aka “columns of data”) have complex data structures).

As soon as you get multiple rows (ie bundles) of data these will be in an array of collections which we will focus on in the next walk through.

In this video we look at how the Make mapping UI (:star:icon) can fail to present the underlying structures so you can easily drag and drop them into the Text Aggregator. The Make mapping UI will show you only the data that passes through the first operation that the Parse JSON module processes. I show you how to use dot notation and type out the correct data reference by hand even though the UI is not letting you pick the necessary element by clicking. This is a handy skill to have!

I also show you how I can make the scenario filter for the variable JSON collection complex values after the iterator, and as a result have Parse JSON module process only 1 operation. This makes it possible to use the Make element mapping UI to click on the elements right into the text aggregator.

I highly encourage you to watch the video because this is probably one of the highest areas where Make users (not just newbies!) get super-confused and ask: Why does the Make data picker UI not show you the elements for me to map?

Download the Make blueprint and use the Import Blueprint to create a new scenario:
4. Managing More Complex Data Structures.json (45.3 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

3 Likes

And now presenting Part 5: Manipulating Data Structures with CSV aggregator and array functions

In this video we introduce the CSV aggregator to create comma separated text files. This aggregator can be very useful when you want to create columnar data from one or more bundles. In this case we convert the variable JSON array of collections complex values into a CSV file. I show you a workaround that may happen to you if you try to stuff a collection into a CSV file. You have to extract the values out of the collection into a set variable and use those instead on the CSV aggregator.

Next we move on to the variable JSON array of collections of really complex values and I show you how to use various array functions like map(), get(), first(), last(), merge() to extract valuable information from an array that has 2 sub arrays of 2 complex data structures. I also show you how to there can be subtle differences in array structures (1 element with multiple key/value pairs in 1 collection vs multiple elements each with their own key/value pair). The subtle difference can cause some grief because the Make UI doesn’t really let you pick the values so you do have to type out the reference using object paths (ie dot notation).

Download the Make blueprint and use the Import Blueprint to create a new scenario:
5. Manipulating Data Structures with CSV aggregator and array functions.json (80.4 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

3 Likes

In Part 6 of the Make for Newbies video scenario walkthrough we will be focusing on Accessing and Merging Collections using map() and iterators.

For this walkthrough we are using the JSON Library Systems data structure. At the top of this structure is a collection called library that has nested arrays to describe the sections of a library and another set of nested arrays in the sections array to describe the books in a library. I highly recommend you study the structure inside the scenario or in the post describing all the data structures I use in this video series.

The data structure is a fairly simple definition of a library and the sections and books housed in the library. I chose this sort of structure because it is probably the closest to a sample real-world structure you may get from a typical API, and there are some interesting things you can do to merge data together and manipulate various arrays and subarrays in the structure.

In the video, I show you how to use map() to extract one library definition from all the libraries as well as how to combine 2 library definitions by merging them into one array. I also show you a potential mistake you could make when trying to retrieve all the books from 2 libraries.

Using iterators is the correct way to extract all the books in all the sections. I use 3 iterators in a row to effectively loop through all the sections of 2 libraries and through all the books in each section in each library. Finally an array aggregator combines all the books into a nice simple flat array of key value pairs for all the information about the books.

Download the Make blueprint and use the Import Blueprint to create a new scenario:

6. Accessing and Merging Collections using map() and Iterators.json (99.6 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

And now…

Part 7: Consolidating Libraries by Combining Catalogs and their Books

In this video walkthrough we reach the summit :mount_fuji: of my work here.

Imagine a town with multiple library branches :bank:, when the city manager :woman_artist: has asked the staff :people_holding_hands: to consolidate the book :books: inventory of multiple branches. This scenario will do a few things:

  1. It allows you to select which libraries you wish to consolidate
  2. It combines the sections of books from all the libraries into 1 catalog
  3. If a book section name overlaps between 2 or more libraries, the books from that section will be consolidated into 1 section with the books from the original sections.
  4. the last Parse JSON module will regenerate a new structure that allows you to confirm the newly consolidated library has been correctly merged with the newly consolidated book sections.
  5. This scenario will work for any number of libraries with any number of sections and books per section per library.

I use the array functions map(), distinct() and the flatten() function in very interesting ways that you may not have considered before so it’s worth a watch just for that. There is also the Transform to JSON module which could be very useful as you undertake more complex operations that require generating JSON from your data objects in Make.

There is an interesting relationship in terms of how many operations are used by this scenario. This sort of analysis can be useful as you begin to estimate the operations cost of doing data manipulation in Make. As with many scenarios there are 2 components of calculating operations usage: operations “overhead” and operations that scale depending on a few factors.

In this case the operations count is based on 6 overhead operations plus the number of libraries to be consolidated and the number of unique section names. So here are some examples operation counts:

  • 2 libraries, 4 unique section names = 6+2+4 = 12 operations
  • 3 libraries, 6 unique section names = 6+3+6 = 15 operations
  • 5 libraries, 8 unique section names = 6+5+8 = 19 operations

This scenario really illustrates how operations are used - no matter how complex the operation an operation is used when a module has to process some data and generate 1 or more bundles, and subsequent modules execute based on the previous modules number of output bundles.

If you place filter between modules, you can of course limit the number of bundles processed and therefore the number of operations used. An important note: filters do not use operations!

In this scenario you will see I have not used any set variables modules although I did use them to determine intermediate values to confirm my work. I removed them, by taking the expressions out of the set variables and inserting them directly in the input mapping for the iterators, reducing the operations spent by 2. This is probably the best way to save operations – remove intermediate set variables when you don’t really need to use them. Any expression will generate an output without using an operation itself so the expression can be entered into say an iterator input box or any of the input boxes for module configuration when the map toggle is turned on.

I recommed you load the scenario into your Make instance and experiment by combining multiple libraries together. You can also modify the variable JSON Library Systems in the very first module and add libraries or add sections to the JSON. Tip: Use JSONLint.com to confirm your structure is correct.

I hope this has Make for Newbies Video Series has proven to be useful for you. There is nothing like working through a few samples to deeply understand data structures and how to manipulate them in Make.

Please leave a comment if you have some ideas on what other topics I could cover in future video walkthroughs.

Download the Make blueprint and use the Import Blueprint to create a new scenario:
7. Consolidating Libraries by Combining Catalogs and their Books.json (102.3 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

I decided to add a Part 8: Examples of using EVERY array function! - Part A

This is Part A where I setup a sample array data structure with several elements with each element containing a collection of key/value pairs. I apply just about every array function in different scenarios against this structure.

Here’s the JSON I parse in Parse JSON in module 1:

[
    {
        "name": "Alex C",
        "age": 53,
        "city": "Toronto"
    },
    {
        "name": "Eric S",
        "age": 43,
        "city": "Hoboken"
    },
    {
        "name": "Liana S"
    },
    {
        "name": "David S"
    }
]

Click this image to open and click again to zoom in:

Here are the expressions used in this Part A of the scenario walkthrough:

Get first array element: {{get(2.array; 1)}}

Get city of second array element: {{get(2.array; "2.city")}}

Get age from first array element using pick: {{pick(get(2.array; 1); "age")}}

Get all cities from each array element using map: {{map(2.array; "city")}}

Get unique city names: {{distinct(map(2.array; "city"))}}

Remove null values: {{remove(map(2.array; "city"); null)}}

This join won’t work. Pass primitive array without keys first!: {{join(2.array)}}

Create string of ages from array, remove empty ages: {{join(remove(map(2.array; "age"); null); "|")}}

Get length of array: {{length(2.array)}}

Get keys of array - must select element!: {{keys(last(2.array))}}

Slice out second collection from array: {{slice(2.array; 1; 2)}}

Merge 1st and 3rd array elements: {{merge(slice(2.array; 0; 1); slice(2.array; 2; 3))}}

Does array contain Hoboken?: {{contains(map(2.array; "city"); "Hoboken")}}

Create a new array with elements: {{add(emptyarray; "some text"; map(2.array; "name"); now)}}

Get names from array, filtered by age: {{map(2.array; "name"; "age"; 53)}}

Randomize name key from all elements of array: {{shuffle(map(2.array; "name"))}}
Randomize all elements of array: {{shuffle(2.array)}}

Sort array by age: {{sort(2.array; "age")}}

Reverse elements of array: {{reverse(2.array)}}

Convert collection to Array: {{toArray(get(2.array; 2))}}

Part B will contain a separate description on using flatten(), deduplicate() and toArray().

Copy and Paste this blueprint into a new Make Scenario
{
    "subflows": [
        {
            "flow": [
                {
                    "id": 1,
                    "module": "json:ParseJSON",
                    "version": 1,
                    "parameters": {
                        "type": ""
                    },
                    "mapper": {
                        "json": "[\n    {\n        \"name\": \"Alex C\",\n        \"age\": 53,\n        \"city\": \"Toronto\"\n    },\n    {\n        \"name\": \"Eric S\",\n        \"age\": 43,\n        \"city\": \"Hoboken\"\n    },\n    {\n        \"name\": \"Liana S\"\n    },\n{\n        \"name\": \"David S\"\n    }\n]"
                    },
                    "metadata": {
                        "designer": {
                            "x": 0,
                            "y": 150
                        },
                        "restore": {
                            "parameters": {
                                "type": {
                                    "label": "Choose a data structure"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "type",
                                "type": "udt",
                                "label": "Data structure"
                            }
                        ],
                        "expect": [
                            {
                                "name": "json",
                                "type": "text",
                                "label": "JSON string",
                                "required": true
                            }
                        ]
                    }
                },
                {
                    "id": 2,
                    "module": "builtin:BasicAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 1
                    },
                    "mapper": {
                        "name": "{{1.name}}",
                        "age": "{{1.age}}",
                        "city": "{{1.city}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 300,
                            "y": 150
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "JSON - Parse JSON [1]"
                                },
                                "target": {
                                    "label": "Custom"
                                }
                            }
                        }
                    }
                },
                {
                    "id": 3,
                    "module": "util:SetVariables",
                    "version": 1,
                    "parameters": {},
                    "mapper": {
                        "variables": [
                            {
                                "name": "Get first array element",
                                "value": "{{get(2.array; 1)}}"
                            },
                            {
                                "name": "Get city of second array element",
                                "value": "{{get(2.array; \"2.city\")}}"
                            },
                            {
                                "name": "Get age from first array element using pick",
                                "value": "{{pick(get(2.array; 1); \"age\")}}"
                            },
                            {
                                "name": "Get all cities from each array element using map",
                                "value": "{{map(2.array; \"city\")}}"
                            },
                            {
                                "name": "Get unique city names",
                                "value": "{{distinct(map(2.array; \"city\"))}}"
                            },
                            {
                                "name": "Remove null values",
                                "value": "{{remove(map(2.array; \"city\"); null)}}"
                            },
                            {
                                "name": "This join won't work. Pass primitive array without keys first!",
                                "value": "{{join(2.array)}}"
                            },
                            {
                                "name": "Create string of ages from array, remove empty ages",
                                "value": "{{join(remove(map(2.array; \"age\"); null); \"|\")}}"
                            },
                            {
                                "name": "Get length of array",
                                "value": "{{length(2.array)}}"
                            },
                            {
                                "name": "Get keys of array - must select element!",
                                "value": "{{keys(last(2.array))}}"
                            },
                            {
                                "name": "Slice out second collection from array",
                                "value": "{{slice(2.array; 1; 2)}}"
                            },
                            {
                                "name": "Merge 1st and 3rd array elements",
                                "value": "{{merge(slice(2.array; 0; 1); slice(2.array; 2; 3))}}"
                            },
                            {
                                "name": "Does array contain Hoboken?",
                                "value": "{{contains(map(2.array; \"city\"); \"Hoboken\")}}"
                            },
                            {
                                "name": "Create a new array with elements",
                                "value": "{{add(emptyarray; \"some text\"; map(2.array; \"name\"); now)}}"
                            },
                            {
                                "name": "Get names from array, filtered by age",
                                "value": "{{map(2.array; \"name\"; \"age\"; 53)}}"
                            },
                            {
                                "name": "Randomize contents of array",
                                "value": "{{shuffle(2.array)}}"
                            },
                            {
                                "name": "Sort array by age",
                                "value": "{{sort(2.array; \"age\")}}"
                            },
                            {
                                "name": "Reverse array of names",
                                "value": "{{reverse(2.array)}}"
                            },
                            {
                                "name": "Convert collection to Array",
                                "value": "{{toArray(get(2.array; 2))}}"
                            }
                        ],
                        "scope": "roundtrip"
                    },
                    "metadata": {
                        "designer": {
                            "x": 600,
                            "y": 150
                        },
                        "restore": {
                            "expect": {
                                "variables": {
                                    "items": [
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null,
                                        null
                                    ]
                                },
                                "scope": {
                                    "label": "One cycle"
                                }
                            }
                        },
                        "expect": [
                            {
                                "name": "variables",
                                "type": "array",
                                "label": "Variables",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Variable name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Variable value",
                                        "type": "any"
                                    }
                                ]
                            },
                            {
                                "name": "scope",
                                "type": "select",
                                "label": "Variable lifetime",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "roundtrip",
                                        "execution"
                                    ]
                                }
                            }
                        ],
                        "interface": [
                            {
                                "name": "Get first array element",
                                "label": "Get first array element",
                                "type": "any"
                            },
                            {
                                "name": "Get city of second array element",
                                "label": "Get city of second array element",
                                "type": "any"
                            },
                            {
                                "name": "Get age from first array element using pick",
                                "label": "Get age from first array element using pick",
                                "type": "any"
                            },
                            {
                                "name": "Get all cities from each array element using map",
                                "label": "Get all cities from each array element using map",
                                "type": "any"
                            },
                            {
                                "name": "Get unique city names",
                                "label": "Get unique city names",
                                "type": "any"
                            },
                            {
                                "name": "Remove null values",
                                "label": "Remove null values",
                                "type": "any"
                            },
                            {
                                "name": "This join won't work. Pass primitive array without keys first!",
                                "label": "This join won't work. Pass primitive array without keys first!",
                                "type": "any"
                            },
                            {
                                "name": "Create string of ages from array, remove empty ages",
                                "label": "Create string of ages from array, remove empty ages",
                                "type": "any"
                            },
                            {
                                "name": "Get length of array",
                                "label": "Get length of array",
                                "type": "any"
                            },
                            {
                                "name": "Get keys of array - must select element!",
                                "label": "Get keys of array - must select element!",
                                "type": "any"
                            },
                            {
                                "name": "Slice out second collection from array",
                                "label": "Slice out second collection from array",
                                "type": "any"
                            },
                            {
                                "name": "Merge 1st and 3rd array elements",
                                "label": "Merge 1st and 3rd array elements",
                                "type": "any"
                            },
                            {
                                "name": "Does array contain Hoboken?",
                                "label": "Does array contain Hoboken?",
                                "type": "any"
                            },
                            {
                                "name": "Create a new array with elements",
                                "label": "Create a new array with elements",
                                "type": "any"
                            },
                            {
                                "name": "Get names from array, filtered by age",
                                "label": "Get names from array, filtered by age",
                                "type": "any"
                            },
                            {
                                "name": "Randomize contents of array",
                                "label": "Randomize contents of array",
                                "type": "any"
                            },
                            {
                                "name": "Sort array by age",
                                "label": "Sort array by age",
                                "type": "any"
                            },
                            {
                                "name": "Reverse array of names",
                                "label": "Reverse array of names",
                                "type": "any"
                            },
                            {
                                "name": "Convert collection to Array",
                                "label": "Convert collection to Array",
                                "type": "any"
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes

Finally we arrive at Part 8: Examples of using EVERY Make array function!: Part B

In Part B, I illustrate the use of the flatten(), deduplicate() and the toCollection() functions. These functions have their own special use and techniques.

We use JSON like this to illustrate flatten() and deduplicate():

{
    "nested-array": [
        "item1",
        "item2",
        "item1",
        "item2",
        [
            "itemsub1",
            "itemsub2",
            "itemsub1",
            [
                "itemsub3-sub1",
                "itemsub3-sub2",
                "itemsub3-sub2",
                "itemsub3-sub3"
            ],
            {
                "itemsub3-coll1": "value1",
                "itemsub3-coll2": "value2",
                "itemsub3-coll3": "value1",
                "itemsub3-coll4": "value3",
                "itemsub3-coll5": "value4",
                "itemsub3-coll6": "value3"
            }
        ]
    ]
}

flatten() serves to collapse nested arrays to an arbitrary depth level. For example, say we have an array like this:

[val1, val2, [val3, val4], val5]

Flattening this array to a depth level of 1 converts the array and, well, flattens it into

[val1, val2, val3, val4, val5]

Let’s add another level of nesting to our working array:

[val1, val2, [val3, [val4, val5]], val6]

Flattening this new array to a depth level of 2 converts the array to a totally flattened array:

[val1, val2, val3, val4, val5, val6]

But flattening the array to a depth level of 1 yields an array with val4 and val5 still nested 1 level (rather than nested 2 levels down):

[val1, val2, val3, [val4, val5], val6]

Using the expression {{flatten(9.nested-array; 2)}} we can flatten the nested-array array in our JSON example to 2 levels. You should study this output carefully and compare to the original array to see how it has been collapsed. Note the collection remains untouched but it has been moved all the way up to the top level of the array.

     "Flatten array 2 levels down": [
            "item1",
            "item2",
            "item1",
            "item2",
            "itemsub1",
            "itemsub2",
            "itemsub1",
            "itemsub3-sub1",
            "itemsub3-sub2",
            "itemsub3-sub2",
            "itemsub3-sub3",
            {
                "itemsub3-coll1": "value1",
                "itemsub3-coll2": "value2",
                "itemsub3-coll3": "value1",
                "itemsub3-coll4": "value3",
                "itemsub3-coll5": "value4",
                "itemsub3-coll6": "value3"
            }
        ]

deduplicate() serves to remove duplicate array values given a particular array. The deduplication returns the object with duplicates removed in the array specified. So if you called deduplicate(nested-array) the top level 4 elements would be deduped leaving only 2 elements, item1 and item2. Note the remaining elements in the array remain untouched.

"Deduplicate array at level 1 ": [
            "item1",
            "item2",
            [
                "itemsub1",
                "itemsub2",
                "itemsub1",
                [
                    "itemsub3-sub1",
                    "itemsub3-sub2",
                    "itemsub3-sub2",
                    "itemsub3-sub3"
                ],
                {
                    "itemsub3-coll1": "value1",
                    "itemsub3-coll2": "value2",
                    "itemsub3-coll3": "value1",
                    "itemsub3-coll4": "value3",
                    "itemsub3-coll5": "value4",
                    "itemsub3-coll6": "value3"
                }
            ]
        ]

You can of course call deduplicate() with one of the nested arrays and the function will return the array at that level deduplicated. Calling {{deduplicate(get(9.nested-array; 5))}} will target the 5th element of nested-array. The result is this array where itemsub1 as a duplicate value has been removed. Note that only the array and all its subitems are returned since we are working directly with the 5th element of the original nested-array.

"Deduplicate array in an array ": [
            "itemsub1",
            "itemsub2",
            [
                "itemsub3-sub1",
                "itemsub3-sub2",
                "itemsub3-sub2",
                "itemsub3-sub3"
            ],
            {
                "itemsub3-coll1": "value1",
                "itemsub3-coll2": "value2",
                "itemsub3-coll3": "value1",
                "itemsub3-coll4": "value3",
                "itemsub3-coll5": "value4",
                "itemsub3-coll6": "value3"
            }
        ]

To illustrate the toArray() function, we can use this expression {{toArray(get(get(9.nested-array; 5); 5))}} which grabs the 5th element of the 5th element of the nested-array object – which is a collection. Note the nested get() functions are a longer way of expressing {{9.nested-array.5.5}} but they mean the same thing. This expression converts the collection to an array. Instead of 6 key value pairs in the original collection, we get an array of 6 elements with the keys key and value holding the original collection’s data.

 "Convert collection into array": [
            {
                "key": "itemsub3-coll1",
                "value": "value1"
            },
            {
                "key": "itemsub3-coll2",
                "value": "value2"
            },
            {
                "key": "itemsub3-coll3",
                "value": "value1"
            },
            {
                "key": "itemsub3-coll4",
                "value": "value3"
            },
            {
                "key": "itemsub3-coll5",
                "value": "value4"
            },
            {
                "key": "itemsub3-coll6",
                "value": "value3"
            }
        ]

The usefulness of the toArray() function comes into focus when we apply deduplicate() and map() on this array to get the unique values of the original collection using the expression {{deduplicate(map(toArray(9.nested-array.5.5); "value"))}}:

"Deduplicate values in a collection": [
            "value1",
            "value2",
            "value3",
            "value4"
        ]

To illustrate the toCollection() function, we now switch to using this JSON array called email:

{
    "email": [
        {
            "fieldname": "Subject",
            "value": "My subject is this!"
        },
        {
            "fieldname": "To",
            "value": "email@email.com"
        },
        {
            "fieldname": "Content",
            "value": "This is the email content."
        }
    ]
}

Converting this array to a collection can be accomplished with the expression {{toCollection(11.email; "fieldname"; "value")}}

Note that the second argument of this call, fieldname, is simply the name of the key whose value will be used to create the new key in the collection. The third argument, value, is the simply the name of the key whose value will be used for the value of the new key in the collection. Study this output bundle to understand how an array gets converted to a collection. Note that if you iterated the original email array you’d get 3 output bundles. Converting it to a collection yields 1 output bundle: a collection with 3 key/value pairs.

[
    {
        "Convert an array to a collection": {
            "Subject": "My subject is this!",
            "To": "email@email.com",
            "Content": "This is the email content."
        }
    }
]

Download the Make blueprint and use the Import Blueprint to create a new scenario:
Part 8 Examples of using EVERY Make array function - part B.json (28.5 KB)


Alex Sirota
Director of NewPath Consulting - we are :superhero: Make Heroes! :woman_superhero:t4:

:heart_on_fire: Check out my series of videos and scenario walkthroughs for Make Newbies :heart_on_fire:

My Solutions on Make Community

4 Likes