How do I clean data in an Array

How To
1

I have an array or almost always have an array where the data is never in the preferred formats. Consequently there is a need to format the entire dataset (not just extracting and formatting a particular field) and returning the same dataset.

I have tried using Iterators and Aggregators (using Create JSON for the target structure), this only works for a typical arrays. It fails where it has a deep nested collections.

As an automation dev, I would love to hear how people deal with this type of problems generally. There is no blueprint, just looking for a POC, which I think is foundational to array manipulation and data cleaning in Make.

3

Hi @Jide_Nubi,

one solution would be the iterator → change each item (potentially with more iterators if there are nested arrays) → aggregate again. This might be credit-intensive as it might consume an operation per item.

Sometimes is good to have a javascript run-time outside (AWS, Google Cloud, Azure, Replit..) or another alternative would be a custom App with some plain Javascript using Make’s IML functions.

Those are my takes on it :slight_smile: Happy to hear other approaches!

Best,
Richard

1 Like
4

Exactly, 10k records will lead to 10k Ops out the window. I am really looking to circumvent that.

I will look into your suggestion around the custom app.
Thanks for your contribution!

1 Like