Transform an array of objects

skanfklwef · January 9, 2024, 9:45pm

Transform a variable length array of objects from a webhook step into another array of objects for use in a HTTP request body
When I use array iterator and array aggregator, I can’t define a custom data structure for the array aggregator to output to (the names need to change).
When I use array iterator and aggregator to JSON, the array iterator cannot map the values inside the packages array

example source array with 2 packages (can vary from 2 to 24)

"packages": [
        {
            "name": "Package 1",
            "weight_in_oz": 4,
            "width": 12,
            "length": 15,
            "height": 4,
            "extra":{"data":"data_value"}
       },
       {
            "name": "Package 2",
            "weight_in_oz": 10,
            "width": 5,
            "length": 5,
            "height": 2,
            "extra":{"data":"data_value"}
       }
]

the “extra” collection and “name” field are dropped (not mapped)

target array to be used in HTTP request body

"shipments":[
				{
					"parcel": {
						"length": "15",
						"width": "12",
						"height": "4",
						"weight": "4"
					},
					"options":{ "option1":"option1_value"}	
				},
				{
					"parcel": {
						"length": "5",
						"width": "5",
						"height": "2",
						"weight": "10"
					},
                    "options":{"option1":"option1_value"}
				}	
]

the “options” collection has pre-defined value so it is not mapped

samliew · January 10, 2024, 1:44am

Welcome to the Make community!

Yes, that is possible. You’ll need a single module:

{{replace(replace(replace(replace(1.value; "/\s*""name"":\s?""[^""]+"",/g"; """parcel"": {"); """weight_in_oz"""; """weight"""); "/,\s*""extra"":\s?{""data"":""[^""]+""}/g"; "},""options"": { ""option1"":""option1_value"" }"); """packages"":"; """shipments"":")}}

Output

Screenshot_2024-01-10_124404

"shipments": [
        {"parcel": {
            "weight": 4,
            "width": 12,
            "length": 15,
            "height": 4},"options": { "option1":"option1_value" }
       },
       {"parcel": {
            "weight": 10,
            "width": 5,
            "length": 5,
            "height": 2},"options": { "option1":"option1_value" }
       }
]

Give it a go and let us know if you have any issues!

skanfklwef · January 10, 2024, 3:06am

Do I need to use array iterator, array aggregator, then Compose a string? Using the Compose a string step right after the webhook generates an error because arrays are not allowed input.

samliew · January 10, 2024, 3:11am

Not if your Webhook is set to JSON pass-through Yes

Perhaps you can provide the full output bundle of your webhook?

skanfklwef · January 10, 2024, 3:13am

[
    {
        "value": "{\n    \"shipping_method\": \"Bluestem_FED_API\",\n    \"order_id\": 242662901,\n    \"profile\": \"bluestem\",\n    \"fulfillment_status\": \"pending\",\n    \"order_number\": \"MO31\",\n    \"shop_name\": \"Manual Order\",\n    \"account_id\": 5927,\n    \"partner_order_id\": \"MO31\",\n    \"shipping_name\": \"Manual Order Shipping Method\",\n    \"tax_type\": null,\n    \"tax_id\": null,\n    \"incoterms\": null,\n    \"currency\": \"USD\",\n    \"from_address\": {\n        \"name\": \"ShipHero\",\n        \"company_name\": \"ShipHero\",\n        \"address_1\": \"123 First St.\",\n        \"address_2\": \"\",\n        \"email\": \"help@shiphero.com\",\n        \"city\": \"Anytown\",\n        \"state\": \"CA\",\n        \"zip\": \"90210\",\n        \"country\": \"US\",\n        \"phone\": \"1234567890\"\n    },\n    \"to_address\": {\n        \"name\": \"Test Order\",\n        \"company_name\": \"\",\n        \"address_1\": \"1 MAIN ST\",\n        \"address_2\": \"\",\n        \"email\": \"help@shiphero.com\",\n        \"city\": \"Anytown\",\n        \"state\": \"CA\",\n        \"zip\": \"90210\",\n        \"country\": \"US\",\n        \"phone\": \"1234567890\"\n    },\n    \"packages\": [\n        {\n            \"name\": \"Package 1\",\n            \"weight_in_oz\": 4,\n            \"width\": 12,\n            \"length\": 15,\n            \"height\": 4,\n            \"line_items\": [\n                {\n                    \"sku\": \"shiphero-socks-black\",\n                    \"tariff_code\": \"00.00.00.000\",\n                    \"price\": 0,\n                    \"customs_description\": \"ShipHero Socks - Black\",\n                    \"customs_value\": \"0.01\",\n                    \"line_item_id\": 639727287,\n                    \"quantity\": 1,\n                    \"weight\": 1.92,\n                    \"partner_line_item_id\": \"MO31-299273207\",\n                    \"id\": \"MO31-299273207\",\n                    \"country_of_manufacture\": \"\",\n                    \"product_name\": \"ShipHero Socks - Black\",\n                    \"name\": \"ShipHero Socks - Black\",\n                    \"ignore_on_customs\": false\n               },\n               {\n                   \"sku\": \"shiphero-socks-blue\",\n                   \"tariff_code\": \"00.00.00.000\",\n                   \"price\": 0,\n                   \"customs_description\": \"ShipHero Socks - Blue\",\n                   \"customs_value\": \"0.01\",\n                   \"line_item_id\": 639727288,\n                   \"quantity\": 1,\n                   \"weight\": 1.92,\n                   \"partner_line_item_id\": \"MO31-299273245\",\n                   \"id\": \"MO31-299273245\",\n                   \"country_of_manufacture\": \"\",\n                   \"product_name\": \"ShipHero Socks - Blue\",\n                   \"name\": \"ShipHero Socks - Blue\",\n                   \"ignore_on_customs\": false\n              }\n        ],\n        \"box_code\": \"custom\",\n        \"box_id\": 5612,\n        \"box_name\": \"12x15 Poly Mailer\"\n    },\n    {\n        \"name\": \"Package 2\",\n        \"weight_in_oz\": 1.08,\n        \"width\": 8,\n        \"length\": 12,\n        \"height\": 2,\n        \"line_items\": [\n            {\n                \"sku\": \"shiphero-hero\",\n                \"tariff_code\": \"00.00.00.000\",\n                \"price\": 0,\n                \"customs_description\": \"ShipHero Hero \",\n                \"customs_value\": \"0.01\",\n                \"line_item_id\": 639727284,\n                \"quantity\": 1,\n                \"weight\": 0.92,\n                \"partner_line_item_id\": \"MO31-299273995\",\n                \"id\": \"MO31-299273995\",\n                \"country_of_manufacture\": \"\",\n                \"product_name\": \"ShipHero Hero \",\n                \"name\": \"ShipHero Hero \",\n                \"ignore_on_customs\": false\n            },\n            {\n                \"sku\": \"shiphero-knit-hat\",\n                \"tariff_code\": \"00.00.00.000\",\n                \"price\": 0,\n                \"customs_description\": \"ShipHero Knit Hat \",\n                \"customs_value\": \"0.01\",\n                \"line_item_id\": 639727285,\n                \"quantity\": 1,\n                \"weight\": 2.19,\n                \"partner_line_item_id\": \"MO31-299273169\",\n                \"id\": \"MO31-299273169\",\n                \"country_of_manufacture\": \"\",\n                \"product_name\": \"ShipHero Knit Hat \",\n                \"name\": \"ShipHero Knit Hat \",\n                \"ignore_on_customs\": false\n            }\n        ],\n        \"box_code\": \"custom\",\n        \"box_id\": 5565,\n        \"box_name\": \"2 x 8 Poly Mailer\"\n       }\n    ]\n}"
    }
]

There are more data in the collection than I need. I only need to keep “weight_in_oz”, “width”, “length”, and “height” from each collection inside package array.

samliew · January 10, 2024, 3:39am

Great, you should always start off your question with the actual data.

Now that it is known that your incoming webhook request data is complex, you’ll need to set JSON pass-through “NO” initially.

You’ll need a minimum of four modules to process the data:

1. Iterate Packages array and Aggregate the fields

2. Transform array to JSON

Screenshot_2024-01-10_143742

3. Compose a String

Screenshot_2024-01-10_143829

Paste this into the field to get the above screenshot

"shipments":{{replace(replace(replace(17.json; "weight_in_oz"; "weight"); "{"; "{""package"":{"); "}"; "},""options"":{""option1"":""option1_value""}}")}}

Replace/remap the variables accordingly.

Output

Screenshot_2024-01-10_144057

Give it a go and let us know if you have any issues!

samliew · January 10, 2024, 3:57am

Here’s another way to do it.

If you still have JSON pass-through “YES”, you can reduce the number of modules by 1

1. Use a Text parser with this regex pattern, AND Global match set to “Yes”

{[\w\W]+?"weight_in_oz":\s?(?<weight>[\d.]+),\s*"width":\s?(?<width>\d+),\s*"length":\s?(?<length>\d+),\s*"height":\s?(?<height>\d+)

Regex test: https://regex101.com/r/rHDd6z

2. Text aggregate each results into your desired JSON, add comma in-between results

3. Wrap the resulting array with your key (shipments)

Output

Screenshot_2024-01-10_145719

skanfklwef · January 10, 2024, 4:51am

Thank you very much for optimizing the solution.

The second solution is producing no output at the text parser.

I copied your regex and enabled Global Match. Did I choose a wrong setting?

samliew · January 10, 2024, 4:55am

The second solution requires JSON Pass-Through “No”.

This is because the Match Pattern module only accepts a string as input.

skanfklwef · January 10, 2024, 4:58am

To clarify, Match Pattern needs webhook JSON passthrough = “No”? I get output as [collection][collection] when set to “No”. When set to “Yes”, no output value.

samliew · January 10, 2024, 5:02am

Sorry, I meant Yes, as mentioned at the top of the post.

Is this what you’re getting from the Webhook when it’s set to Pass-through Yes?

If yes, you have to “Parse JSON” one time before you can use the value variable in the Text Parser module.

skanfklwef · January 10, 2024, 5:13am

Yes, that is what I get when Pass-through = “Yes”. Adding a Parse JSON step before the text parser gives error: missing value of required parameter ‘json’. I had Make.com automatically determine the data structure.

samliew · January 10, 2024, 5:31am

This is what it should look like

To further assist you, I’ll need the input and output bundles of your Text Parser module.

skanfklwef · January 10, 2024, 5:35am

Text Parser input:

[
    {
        "text": "{\n    \"shipping_method\": \"Bluestem_FED_API\",\n    \"order_id\": 242662901,\n    \"profile\": \"bluestem\",\n    \"fulfillment_status\": \"pending\",\n    \"order_number\": \"MO31\",\n    \"shop_name\": \"Manual Order\",\n    \"account_id\": 5927,\n    \"partner_order_id\": \"MO31\",\n    \"shipping_name\": \"Manual Order Shipping Method\",\n    \"tax_type\": null,\n    \"tax_id\": null,\n    \"incoterms\": null,\n    \"currency\": \"USD\",\n    \"from_address\": {\n        \"name\": \"ShipHero\",\n        \"company_name\": \"ShipHero\",\n        \"address_1\": \"123 First St.\",\n        \"address_2\": \"\",\n        \"email\": \"help@shiphero.com\",\n        \"city\": \"Anytown\",\n        \"state\": \"CA\",\n        \"zip\": \"90210\",\n        \"country\": \"US\",\n        \"phone\": \"1234567890\"\n    },\n    \"to_address\": {\n        \"name\": \"Test Order\",\n        \"company_name\": \"\",\n        \"address_1\": \"1 MAIN ST\",\n        \"address_2\": \"\",\n        \"email\": \"help@shiphero.com\",\n        \"city\": \"Anytown\",\n        \"state\": \"CA\",\n        \"zip\": \"90210\",\n        \"country\": \"US\",\n        \"phone\": \"1234567890\"\n    },\n    \"packages\": [\n        {\n            \"name\": \"Package 1\",\n            \"weight_in_oz\": 4,\n            \"width\": 12,\n            \"length\": 15,\n            \"height\": 4,\n            \"line_items\": [\n                {\n                    \"sku\": \"shiphero-socks-black\",\n                    \"tariff_code\": \"00.00.00.000\",\n                    \"price\": 0,\n                    \"customs_description\": \"ShipHero Socks - Black\",\n                    \"customs_value\": \"0.01\",\n                    \"line_item_id\": 639727287,\n                    \"quantity\": 1,\n                    \"weight\": 1.92,\n                    \"partner_line_item_id\": \"MO31-299273207\",\n                    \"id\": \"MO31-299273207\",\n                    \"country_of_manufacture\": \"\",\n                    \"product_name\": \"ShipHero Socks - Black\",\n                    \"name\": \"ShipHero Socks - Black\",\n                    \"ignore_on_customs\": false\n               },\n               {\n                   \"sku\": \"shiphero-socks-blue\",\n                   \"tariff_code\": \"00.00.00.000\",\n                   \"price\": 0,\n                   \"customs_description\": \"ShipHero Socks - Blue\",\n                   \"customs_value\": \"0.01\",\n                   \"line_item_id\": 639727288,\n                   \"quantity\": 1,\n                   \"weight\": 1.92,\n                   \"partner_line_item_id\": \"MO31-299273245\",\n                   \"id\": \"MO31-299273245\",\n                   \"country_of_manufacture\": \"\",\n                   \"product_name\": \"ShipHero Socks - Blue\",\n                   \"name\": \"ShipHero Socks - Blue\",\n                   \"ignore_on_customs\": false\n              }\n        ],\n        \"box_code\": \"custom\",\n        \"box_id\": 5612,\n        \"box_name\": \"12x15 Poly Mailer\"\n    },\n    {\n        \"name\": \"Package 2\",\n        \"weight_in_oz\": 1.08,\n        \"width\": 8,\n        \"length\": 12,\n        \"height\": 2,\n        \"line_items\": [\n            {\n                \"sku\": \"shiphero-hero\",\n                \"tariff_code\": \"00.00.00.000\",\n                \"price\": 0,\n                \"customs_description\": \"ShipHero Hero \",\n                \"customs_value\": \"0.01\",\n                \"line_item_id\": 639727284,\n                \"quantity\": 1,\n                \"weight\": 0.92,\n                \"partner_line_item_id\": \"MO31-299273995\",\n                \"id\": \"MO31-299273995\",\n                \"country_of_manufacture\": \"\",\n                \"product_name\": \"ShipHero Hero \",\n                \"name\": \"ShipHero Hero \",\n                \"ignore_on_customs\": false\n            },\n            {\n                \"sku\": \"shiphero-knit-hat\",\n                \"tariff_code\": \"00.00.00.000\",\n                \"price\": 0,\n                \"customs_description\": \"ShipHero Knit Hat \",\n                \"customs_value\": \"0.01\",\n                \"line_item_id\": 639727285,\n                \"quantity\": 1,\n                \"weight\": 2.19,\n                \"partner_line_item_id\": \"MO31-299273169\",\n                \"id\": \"MO31-299273169\",\n                \"country_of_manufacture\": \"\",\n                \"product_name\": \"ShipHero Knit Hat \",\n                \"name\": \"ShipHero Knit Hat \",\n                \"ignore_on_customs\": false\n            }\n        ],\n        \"box_code\": \"custom\",\n        \"box_id\": 5565,\n        \"box_name\": \"2 x 8 Poly Mailer\"\n       }\n    ]\n}"
    }
]

Text Parser Output:

[]

Scenario:

samliew · January 10, 2024, 5:53am

The only things I can think of, is the pattern need to be copied exactly the same, with no leading or trailing spaces.

Try coping the pattern directly from the link instead https://regex101.com/r/rHDd6z

If this doesn’t work, either use the first method, or you might have to invite me into your Team to take a look.

skanfklwef · January 10, 2024, 6:09am

Thanks for the tip! I trimmed the regex and it is working now. However, I have a new problem: JSON passthrough appears to break the mapping in the subsequent HTTP module for other fields. So if I need other fields to map to subsequent steps, I need to modify the regex to match more patterns?

samliew · January 10, 2024, 7:32am

If you need other mappings other than just the packages, then the first option with JSON pass-through may be more appropriate.

Although you can just Parse JSON on the original Webhook value.

skanfklwef · January 10, 2024, 7:34am

Thank you once again, I’ve learned a lot from you.

samliew · January 10, 2024, 10:38am

No problem, glad I could help!

Links

Here are some useful links and guides if you would like to learn more on how to use the Make platform, apps, and app modules —

General

Help Center | Tutorials
Make Academy – learn Make and get your certificate
Make Blog – get the latest updates
Features & Pricing

Help Center Basics

Mapping – What is mapping? What can I map?
Mapping with arrays – How to map items in an array
Tokens for parseDate | Tokens for formatDate
HTTP modules – Make a request, Get (download) a file
Webhooks – Error Handling, Responding to webhooks

Articles & Videos

skanfklwef · January 10, 2024, 7:49pm

@samliew I have another question. I’m using a router at the beginning of the scenario to branch the scenario depending on the length of the packages array. Since len(packages)=1 is more common, should I just disable json passthrough to make mapping easier and use Transform to JSON and Parse JSON before Text Parser? I’ve attached the blueprint.

I had an issue with the first solution you posted since the webhook passed dimension values as number but I need to convert them to string, the replace function couldn’t fix that.

Topic		Replies	Views
Transform Webhook array for use in HTTP request How To	2	563	December 7, 2023
How to transform a json to another? How To arrays	6	27	October 11, 2024
How do I map all items from an array value into JSON format? How To arrays , json	3	2934	January 27, 2024
Transform an array with an array that contents collections into a new json How To arrays , collections	3	333	June 20, 2024
Mapping collection to collection with different keys How To collections	3	51	November 3, 2024

Transform an array of objects

example source array with 2 packages (can vary from 2 to 24)

target array to be used in HTTP request body

Output

1. Iterate Packages array and Aggregate the fields

2. Transform array to JSON

3. Compose a String

Output

1. Use a Text parser with this regex pattern, AND Global match set to “Yes”

2. Text aggregate each results into your desired JSON, add comma in-between results

3. Wrap the resulting array with your key (shipments)

Output

Links

General

Help Center Basics

Articles & Videos

Related Topics