Sum occurrences in a nested array with collections

Dear community,

I am relatively new to make.com and hope you can help me with the following question:

I request an API via an HTTP module and get the following result

I would now like to count how often each email address appears in the result. Unfortunately, the email address is somewhat nested on the second level of a collection.

Result would be sth like:

Mail1 → 10 appearances
Mail 2 → 0 appearances

Based on the number of appearances i need to do more logic then ( do sth when number bigger that 50 e.g)

How would i achieve that? Thank you very much.

Best regards

Welcome to the Make community!

Please provide the output bundles of the modules by running the scenario (or get from the scenario History tab), then click the white speech bubble on the top-right of each module and select “Download input/output bundles”.
Screenshot_2023-10-06_141025

A.

Save each bundle contents in your text editor as a bundle.txt file, and upload it here into this discussion thread.

Uploading them here will look like this:

module-1-output-bundle.txt (12.3 KB)

B.

If you are unable to upload files on this forum, alternatively you can paste the formatted bundles in this manner:

  • Either add three backticks ``` before and after the code, like this:

    ```
    input/output bundle content goes here
    ```

  • Or use the format code button in the editor:
    Screenshot_2023-10-02_191027

If there are any sensitive data in the output bundles, feel free to modify them, BUT keeping the same format/structure of the JSON.

Providing the input/output bundles will allow others to replicate what is going on in the scenario even if they do not use the external service.

This will allow others to better assist you. Thanks!

2 Likes

output_bundle.txt (9.8 KB)

Welcome to the Make community!

Yes, that is possible. You’ll need a minimum of four modules:

The secret is “Parse Response” in the HTTP module must be set to No (default).

Screenshot_2024-04-23_230426

1. Text Parser

You can use a Text Parser “Match Pattern” module with this Pattern (regular expression):

"(?<email>[^"@]+@[^"]+)"

Proof

https://regex101.com/r/PdPhjk

Important Info

  • :warning: Global match must be set to YES!

2. Array Aggregator

Then, another secret is the Array Aggregator to group by email (match)

3. Text Aggregator (if necessary)

To get the number of email matches, you can use the built-in function length.

Screenshot_2024-04-24_000447

Output

Screenshot_2024-04-24_000428

Give it a go and let us know if you have any issues!

samliewrequest private consultation

2 Likes

Module Export

You can copy and paste this module export into your scenario. This will paste the modules shown in my screenshots above.

  1. Copy the JSON code below by clicking the copy button when you mouseover the top-right of the code block
    Screenshot_2024-01-17_200117

  2. Enter your scenario editor. Press ESC to close any dialogs. Press CTRLV (paste keyboard shortcut for Windows) to paste directly in the canvas.

  3. Click on each imported module and save it for validation. You may be prompted to remap some variables and connections.

JSON

{
    "subflows": [
        {
            "flow": [
                {
                    "id": 39,
                    "module": "http:ActionSendData",
                    "version": 3,
                    "parameters": {
                        "handleErrors": true,
                        "useNewZLibDeCompress": true
                    },
                    "mapper": {
                        "url": "https://google.com",
                        "serializeUrl": false,
                        "method": "get",
                        "headers": [],
                        "qs": [],
                        "bodyType": "",
                        "parseResponse": false,
                        "authUser": "",
                        "authPass": "",
                        "timeout": "",
                        "shareCookies": false,
                        "ca": "",
                        "rejectUnauthorized": true,
                        "followRedirect": true,
                        "useQuerystring": false,
                        "gzip": true,
                        "useMtls": false,
                        "followAllRedirects": false
                    },
                    "metadata": {
                        "designer": {
                            "x": -1098,
                            "y": -922
                        },
                        "restore": {
                            "expect": {
                                "method": {
                                    "mode": "chose",
                                    "label": "GET"
                                },
                                "headers": {
                                    "mode": "chose"
                                },
                                "qs": {
                                    "mode": "chose"
                                },
                                "bodyType": {
                                    "label": "Empty"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "handleErrors",
                                "type": "boolean",
                                "label": "Evaluate all states as errors (except for 2xx and 3xx )",
                                "required": true
                            },
                            {
                                "name": "useNewZLibDeCompress",
                                "type": "hidden"
                            }
                        ],
                        "expect": [
                            {
                                "name": "url",
                                "type": "url",
                                "label": "URL",
                                "required": true
                            },
                            {
                                "name": "serializeUrl",
                                "type": "boolean",
                                "label": "Serialize URL",
                                "required": true
                            },
                            {
                                "name": "method",
                                "type": "select",
                                "label": "Method",
                                "required": true,
                                "validate": {
                                    "enum": [
                                        "get",
                                        "head",
                                        "post",
                                        "put",
                                        "patch",
                                        "delete",
                                        "options"
                                    ]
                                }
                            },
                            {
                                "name": "headers",
                                "type": "array",
                                "label": "Headers",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Value",
                                        "type": "text"
                                    }
                                ]
                            },
                            {
                                "name": "qs",
                                "type": "array",
                                "label": "Query String",
                                "spec": [
                                    {
                                        "name": "name",
                                        "label": "Name",
                                        "type": "text",
                                        "required": true
                                    },
                                    {
                                        "name": "value",
                                        "label": "Value",
                                        "type": "text"
                                    }
                                ]
                            },
                            {
                                "name": "bodyType",
                                "type": "select",
                                "label": "Body type",
                                "validate": {
                                    "enum": [
                                        "raw",
                                        "x_www_form_urlencoded",
                                        "multipart_form_data"
                                    ]
                                }
                            },
                            {
                                "name": "parseResponse",
                                "type": "boolean",
                                "label": "Parse response",
                                "required": true
                            },
                            {
                                "name": "authUser",
                                "type": "text",
                                "label": "User name"
                            },
                            {
                                "name": "authPass",
                                "type": "password",
                                "label": "Password"
                            },
                            {
                                "name": "timeout",
                                "type": "uinteger",
                                "label": "Timeout",
                                "validate": {
                                    "max": 300,
                                    "min": 1
                                }
                            },
                            {
                                "name": "shareCookies",
                                "type": "boolean",
                                "label": "Share cookies with other HTTP modules",
                                "required": true
                            },
                            {
                                "name": "ca",
                                "type": "cert",
                                "label": "Self-signed certificate"
                            },
                            {
                                "name": "rejectUnauthorized",
                                "type": "boolean",
                                "label": "Reject connections that are using unverified (self-signed) certificates",
                                "required": true
                            },
                            {
                                "name": "followRedirect",
                                "type": "boolean",
                                "label": "Follow redirect",
                                "required": true
                            },
                            {
                                "name": "useQuerystring",
                                "type": "boolean",
                                "label": "Disable serialization of multiple same query string keys as arrays",
                                "required": true
                            },
                            {
                                "name": "gzip",
                                "type": "boolean",
                                "label": "Request compressed content",
                                "required": true
                            },
                            {
                                "name": "useMtls",
                                "type": "boolean",
                                "label": "Use Mutual TLS",
                                "required": true
                            },
                            {
                                "name": "followAllRedirects",
                                "type": "boolean",
                                "label": "Follow all redirect",
                                "required": true
                            }
                        ],
                        "advanced": true
                    }
                },
                {
                    "id": 34,
                    "module": "regexp:Parser",
                    "version": 1,
                    "parameters": {
                        "pattern": "\"(?<email>[^\"@]+@[^\"]+)\"",
                        "global": true,
                        "sensitive": true,
                        "multiline": false,
                        "singleline": false,
                        "continueWhenNoRes": false,
                        "ignoreInfiniteLoopsWhenGlobal": false
                    },
                    "mapper": {
                        "text": "{{38.value}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": -857,
                            "y": -918,
                            "messages": [
                                {
                                    "category": "reference",
                                    "severity": "warning",
                                    "message": "Referenced module 'Tools - Compose a string' [38] is not accessible."
                                }
                            ]
                        },
                        "restore": {},
                        "parameters": [
                            {
                                "name": "pattern",
                                "type": "text",
                                "label": "Pattern",
                                "required": true
                            },
                            {
                                "name": "global",
                                "type": "boolean",
                                "label": "Global match",
                                "required": true
                            },
                            {
                                "name": "sensitive",
                                "type": "boolean",
                                "label": "Case sensitive",
                                "required": true
                            },
                            {
                                "name": "multiline",
                                "type": "boolean",
                                "label": "Multiline",
                                "required": true
                            },
                            {
                                "name": "singleline",
                                "type": "boolean",
                                "label": "Singleline",
                                "required": true
                            },
                            {
                                "name": "continueWhenNoRes",
                                "type": "boolean",
                                "label": "Continue the execution of the route even if the module finds no matches",
                                "required": true
                            },
                            {
                                "name": "ignoreInfiniteLoopsWhenGlobal",
                                "type": "boolean",
                                "label": "Ignore errors when there is an infinite search loop",
                                "required": true
                            }
                        ],
                        "expect": [
                            {
                                "name": "text",
                                "type": "text",
                                "label": "Text"
                            }
                        ],
                        "interface": [
                            {
                                "type": "text",
                                "name": "email",
                                "label": "email"
                            },
                            {
                                "type": "uinteger",
                                "name": "i",
                                "label": "i"
                            },
                            {
                                "type": "any",
                                "name": "__IMTMATCH__",
                                "label": "Fallback Match"
                            }
                        ]
                    }
                },
                {
                    "id": 35,
                    "module": "builtin:BasicAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 34
                    },
                    "mapper": {
                        "email": "{{34.email}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": -614,
                            "y": -919
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "Text parser - Match pattern [34]"
                                },
                                "target": {
                                    "label": "Custom"
                                }
                            }
                        },
                        "advanced": true
                    },
                    "flags": {
                        "groupBy": "{{34.email}}"
                    }
                },
                {
                    "id": 36,
                    "module": "util:TextAggregator",
                    "version": 1,
                    "parameters": {
                        "rowSeparator": "\n",
                        "feeder": 35
                    },
                    "mapper": {
                        "value": "The count of {{35.`__IMTKEY__`}} is {{length(35.array)}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": -368,
                            "y": -920,
                            "messages": [
                                {
                                    "category": "last",
                                    "severity": "warning",
                                    "message": "A transformer should not be the last module in the route."
                                }
                            ]
                        },
                        "restore": {
                            "parameters": {
                                "rowSeparator": {
                                    "label": "New row"
                                }
                            },
                            "extra": {
                                "feeder": {
                                    "label": "Array aggregator [35]"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "rowSeparator",
                                "type": "select",
                                "label": "Row separator",
                                "validate": {
                                    "enum": [
                                        "\n",
                                        "\t",
                                        "other"
                                    ]
                                }
                            }
                        ],
                        "expect": [
                            {
                                "name": "value",
                                "type": "text",
                                "multiline": true,
                                "label": "Text"
                            }
                        ],
                        "advanced": true
                    }
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}

samliewrequest private consultation

Wow! Amazing how accurate and well document this answer is. Thanks a lot!

I works perfect for me, only i do not understand all of the steps 100% but will try to do now.

Don’t want to take too much of you time, but if possible one last question: If i want to sent a mal to only mails occure more then e.g. 10 times, how would i do that?

THX a lot.

You will add a filter after the array aggregator.

length(array)

Numeric greater than or equal to

10

Follow length(array) from my screenshot as function and variable like this

Screenshot_2024-04-24_000447

No problem, glad I could help!

1. If you have a new question in the future, please start a new thread. This makes it easier for others with the same problem to search for the answers to specific questions, and you are more likely to receive help since newer questions are monitored closely.

2. The Make Community guidelines encourages users to try to mark helpful replies as solutions to help keep the Community organized.

This marks the topic as solved, so that:

others can save time when catching up with the latest activity here, and

  • allows others to quickly jump to the solution if they come across the same problem

To do this, simply click the checkbox at the bottom of the post that answers your question:
Screenshot_2023-10-04_161049

3. Don’t forget to like and bookmark this topic so you can get back to it easily in future!

samliewrequest private consultation

3 Likes

Will do, thank you very much.

1 Like

Hi Dan,

Samliew has already given a great and efficient way to achieve this, another option is the map() function which is a common and powerful way to manipulate data in Make, you could have the business logic you want before the array aggregator.

There may be a more versatile approach using the map() & distinct() functions, it allows you to pick out values in the data structure to work with, keeping them structured will make life easier to follow-on logic.

So using the Data field from your example and seeing that the e-mails are under profil then profilEmail we could address them as profil.profilEmail so we would use map({{??.data}};profil.profilEmail) to get all the values in returned in the API response, to turn that into a list of unique e-mails we would then use distinct() so it would be distinct(map({{??.data}};profil.profilEmail)) which will give us back an array of unique e-mail addresses.

Now the great thing is that you can do that inside the iterator to save an operation, then we can iterate through the API response using the unique list of e-mails and count them up.

I have attached a recording example, in the example module #2 would be your HTTP module so we only need three modules.

For example a router when e-mail count is over a given value as shown in the following (example used 4) is nice and simple, as would be other more complex logic.

The strength of the map comes into play more if you wanted to add additional fields or where there may be more than one regex match for a field on each entry as it can be far easier to maintain parameters by their location and not have to create increasingly complex regex.

It also reduces the risk of errors that can be missed in the regex, for example (?<email>[^"@]+@[^"]+) will match other fields that have @ in them, so if profilVorname contained an @ then you would have that treated as a unique e-mail in your count (entirely fixable with more complex regex). You can see this in the following recording where my payload has allowed a fictional user to include a special character.

To be clear Samliew’s solution is super efficient, this will use more operations based on unique e-mails but is potentially a safer and more versatile approach you may find handy.

4 Likes

Hi bpt_scodon,

appreciate your help a lot. Again stuck how helpful this community is! But samliew’s solution works fine for me plus saves operations. So i already go with it. But THX

1 Like