Structure flat webhook request to be able to iterate over it

I am connecting an old webshop system to a Make.com webhook.

When a new order is placed a request is send to the webhook URL.

The request contains data about the customer and order.

The order data contains the cart items (Artikel in german).

The webhook request looks like this:

The goal:

Structure the cart items (Artikel) to a structure, so I can iterate over them.

The problem

The webhook request bundle contains the data of the cart items like:


Artikelname_1
ArtikelMenge_1
ArtikelEPreis_1
Artikelname_2
ArtikelMenge_2
ArtikelEPreis_2

So not nested, but all in the same level, connected by an index.

What you normally would expect is a nested structure like:

Artikel: {
[
name: “”,
menge: “”,
preis: “”
]
}

But I cannot adjust the webshop side from the where the request is sent.

So now I need to find a solution to iterate over the elements in the webhook request bundle.

Does anyone know how to structure this?

@vyavasthapak That looks like some random output from ChatGPT, and does not answer the question at all. You cannot use ChatGPT to answer questions on this forum, because ChatGPT has no knowledge of Make’s capabilities, functions, apps, and modules.


@Robbertv Welcome to the Make community!

Please provide the output bundles of that Webhook module by running the scenario, then click the white speech bubble on the top-right of each module and select “Download output bundles”.
Screenshot_2023-10-06_141025

A.

Save the bundle contents in your text editor as a bundle.txt file, and upload it here into this discussion thread.
Uploading it here will look like this:

bundle.txt (12.3 KB)

B.

If you are unable to upload files on this forum, alternatively you can paste the formatted output bundle in this manner:

  • Either add three backticks ``` before and after the code, like this:

    ```
    input/output bundle content goes here
    ```

  • Or use the format code button in the editor:
    Screenshot_2023-10-02_191027

Providing the output bundles will allow others to replicate what is going on in the scenario even if they do not use the external service.

This will allow others to better assist you. Thanks!

2 Likes

Hi @samliew,

Thanks for your answer and welcoming me.

The output bundles is:

[
    {
        "Kemail": "johan@test.nl",
        "KLVorname": "Johan",
        "KNachname": "Test",
        "KLFirma": "Company 123",
        "KLStrasse": "Kalverstraat",
        "PosAnz": "1058DC",
        "KTelefon": "+31610101010",
        "VID": "123456789",
        "ArtikelEPreis_1": "853,93",
        "ArtikelEPreis_2": "70,64",
        "Artikelname_1": "LED Mirror Portau",
        "Artikelname_2": "Spaanplaat ruw",
        "ArtikelMenge_1": "1",
        "ArtikelMenge_2": "3"
    }
]

Welcome to the Make community!

Yes, that is possible. You’ll need a minimum of four modules:

0. Set Webhook to Pass-through

You can do that by setting the Webhook “JSON Pass-through” to YES (it is “No” by default), when creating or editing a webhook.

Then, you will be able to map the entire JSON body payload in another module.

1. Match pattern in your webhook JSON

Use a Text Parser “Match Pattern” module with this regular expression pattern

"(?<key>[^"]+)_(?<number>\d)": "?(?<value>[^"]+)(?:"|,)+\s

Regex test: https://regex101.com/r/34Y18M

Important Info

  • :warning: Global match must be set to YES!

2. Aggregate to Text #1 with   "key":"value", group by item number

3. Aggregate to Text #2 with   {}  object wrapper

4. Parse JSON with array wrapper

You can copy this so you don’t have to type it out, insert variable between square brackets

{"items":[ ]}

Output

Screenshot_2024-01-17_200106

Give it a go and let us know if you have any issues!

3 Likes

You can copy and paste the module export in your scenario to import the modules shown in my screenshots above.

  1. Copy the code below by clicking the copy button when you mouseover the top-right of the code block
    Screenshot_2024-01-17_200117

  2. Enter your scenario editor. Press ESC to close any dialogs. Press CTRLV to paste in the canvas.

  3. Click on each imported module and save it. You may need to remap some variables.

Modules JSON Export

{
    "subflows": [
        {
            "flow": [
                {
                    "id": 7,
                    "module": "regexp:Parser",
                    "version": 1,
                    "parameters": {
                        "global": true,
                        "pattern": "\"(?<key>[^\"]+)_(?<number>\\d)\": \"?(?<value>[^\"]+)(?:\"|,)+\\s",
                        "multiline": false,
                        "sensitive": true,
                        "singleline": false,
                        "continueWhenNoRes": true,
                        "ignoreInfiniteLoopsWhenGlobal": false
                    },
                    "mapper": {
                        "text": "{{13.value}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 241,
                            "y": -348
                        },
                        "restore": {},
                        "parameters": [
                            {
                                "name": "pattern",
                                "type": "text",
                                "label": "Pattern",
                                "required": true
                            },
                            {
                                "name": "global",
                                "type": "boolean",
                                "label": "Global match",
                                "required": true
                            },
                            {
                                "name": "sensitive",
                                "type": "boolean",
                                "label": "Case sensitive",
                                "required": true
                            },
                            {
                                "name": "multiline",
                                "type": "boolean",
                                "label": "Multiline",
                                "required": true
                            },
                            {
                                "name": "singleline",
                                "type": "boolean",
                                "label": "Singleline",
                                "required": true
                            },
                            {
                                "name": "continueWhenNoRes",
                                "type": "boolean",
                                "label": "Continue the execution of the route even if the module finds no matches",
                                "required": true
                            },
                            {
                                "name": "ignoreInfiniteLoopsWhenGlobal",
                                "type": "boolean",
                                "label": "Ignore errors when there is an infinite search loop",
                                "required": true
                            }
                        ],
                        "expect": [
                            {
                                "name": "text",
                                "type": "text",
                                "label": "Text"
                            }
                        ],
                        "interface": [
                            {
                                "name": "key",
                                "type": "text",
                                "label": "key"
                            },
                            {
                                "name": "number",
                                "type": "text",
                                "label": "number"
                            },
                            {
                                "name": "value",
                                "type": "text",
                                "label": "value"
                            },
                            {
                                "name": "i",
                                "type": "uinteger",
                                "label": "i"
                            },
                            {
                                "name": "__IMTMATCH__",
                                "type": "any",
                                "label": "Fallback Match"
                            }
                        ]
                    }
                },
                {
                    "id": 15,
                    "module": "util:TextAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 7,
                        "rowSeparator": "other",
                        "otherRowSeparator": ","
                    },
                    "mapper": {
                        "value": "\"{{7.key}}\": \"{{7.value}}\""
                    },
                    "metadata": {
                        "designer": {
                            "x": 484,
                            "y": -347
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "Text parser - Match pattern [7]"
                                }
                            },
                            "parameters": {
                                "rowSeparator": {
                                    "label": "Other"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "rowSeparator",
                                "type": "select",
                                "label": "Row separator",
                                "validate": {
                                    "enum": [
                                        "\n",
                                        "\t",
                                        "other"
                                    ]
                                }
                            },
                            {
                                "name": "otherRowSeparator",
                                "type": "text",
                                "label": "Separator"
                            }
                        ],
                        "expect": [
                            {
                                "name": "value",
                                "type": "text",
                                "label": "Text"
                            }
                        ],
                        "advanced": true
                    },
                    "flags": {
                        "groupBy": "{{7.number}}"
                    }
                },
                {
                    "id": 16,
                    "module": "util:TextAggregator",
                    "version": 1,
                    "parameters": {
                        "feeder": 15,
                        "rowSeparator": "other",
                        "otherRowSeparator": ","
                    },
                    "mapper": {
                        "value": "{{{15.text}}}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 733,
                            "y": -347
                        },
                        "restore": {
                            "extra": {
                                "feeder": {
                                    "label": "Tools - Text aggregator [15]"
                                }
                            },
                            "parameters": {
                                "rowSeparator": {
                                    "label": "Other"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "rowSeparator",
                                "type": "select",
                                "label": "Row separator",
                                "validate": {
                                    "enum": [
                                        "\n",
                                        "\t",
                                        "other"
                                    ]
                                }
                            },
                            {
                                "name": "otherRowSeparator",
                                "type": "text",
                                "label": "Separator"
                            }
                        ],
                        "expect": [
                            {
                                "name": "value",
                                "type": "text",
                                "label": "Text"
                            }
                        ],
                        "advanced": true
                    }
                },
                {
                    "id": 17,
                    "module": "json:ParseJSON",
                    "version": 1,
                    "parameters": {
                        "type": ""
                    },
                    "mapper": {
                        "json": "{ \"items\": [{{16.text}}]}"
                    },
                    "metadata": {
                        "designer": {
                            "x": 980,
                            "y": -345
                        },
                        "restore": {
                            "parameters": {
                                "type": {
                                    "label": "Choose a data structure"
                                }
                            }
                        },
                        "parameters": [
                            {
                                "name": "type",
                                "type": "udt",
                                "label": "Data structure"
                            }
                        ],
                        "expect": [
                            {
                                "name": "json",
                                "type": "text",
                                "label": "JSON string",
                                "required": true
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "metadata": {
        "version": 1
    }
}
2 Likes

Thank you very much for the super detailed answer!

I think I forgot something important to mention:

The webshop makes a PHP curl request like this:

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $webhook_url);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

// A string containing all data
$DATAstring = "......ArtikelEPreis_2=153,5100&ArtikelMwst_2=21,0000&ArtikelMenge_2=1&Artikelname_3=Kupon&ArtikelEPreis_3=-357,01&ArtikelMwst_3=21,0000&ArtikelMenge_3=1&PosAnz=3&Versandkosten=0&kommentar=&Versandart=Verzendkosten%20&NoVersandCalc=1&Zahlart=&NoFeedback=0.......";

curl_setopt($ch, CURLOPT_POSTFIELDS, $DATAstring);
curl_close($ch);

To test quickly I tried to reproduce this with a Postman form-data request.

But the actual request is a curl request with a data string, which Make - when receiving - automatically converts into a bundle.

In your explanation you convert the webhook data into a string, but the actual request is already a string of data, only Make automatically converts it into a bundle.

So in a nutshel; the original request is not a JSON body, but a data string (or form-data when testing with Postman)

So I think I might need another start with the “JSON Pass-through” setting, correct?

You still can pass-through, just that it won’t be JSON, and the new regex patten will be

(?<key>[^=]+)_(?<number>\d)=(?<value>[^&]+)

https://regex101.com/r/fGrZ5c

2 Likes

Ok, but I cannot use the data string from the webshook in the parser module:

This is how it reaches the webhook module:

And then this is the parser:

Do you have the JSON pass-through setting set to yes? :thinking:

2 Likes

Yes I guess so…

Then I’m out of ideas. If the web page is under your control, see if you can send JSON instead.

1 Like

Ok at least thank for your time and effort.

I now do have a value, but it is a buffer instead of a string.

I already tried to convert it to a string inside the parser. But does not seem to work:

The buffer value is probably nothing, it says its length is 0.

1 Like