Transform API Response from Collection of Collections to Array of Collections for Custom Module

Hi Makers,

I’m experimenting with creating a custom module (App) that consumes the Alpha Vantage API.

The API’s “Intraday” time series route returns its data points as structure of nested collections - here’s a link to their demo call. I’m not a fan of how they’re structuring the response for this route because

  1. Each data point is structured identically, so I think they should have been returned as an array of collections rather than a nested set of collections.
  2. They’ve assigned each sub-collection a key containing the date/time of the data point - I think it would have been better to return “dateTime”: “2022-07-14 18:45:00”.

I’d like to abstract away these idiosyncrasies in my module to make the data easier to consume in downstream modules. Ideally I’m trying to make the module return one bundle containing a collection for the metadata returned from the route and an array of collections containing the time series data points’ dateTime, open, high, low, close, and volume.

I tried working in the Interface tab of the module, and also played around with the “output” setting in Communication. I also looked into the IML function toArray but couldn’t figure out where I might inject this to turn the time series nested collection into an array of collections.

It would be great if you could point me in the right direction for how to solve this! :smiley:

Thank you!
John

Hi John, could you share an example output from the API? That might help us provide some direction quickly.

Thanks for the response Darin! Of course - attached is the output from the API call. It’s also available via this live demo call from the API: https://www.alphavantage.co/query?function=TIME_SERIES_INTRADAY&symbol=IBM&interval=5min&apikey=demo

response.json (22.3 KB)

As is, my custom module is producing output in this structure:

Bundle 1 (collection)
   Meta Data (collection)
   Time Series (collection)
      2022-07-14 18:45:00 (collection)
      2022-07-14 16:25:00 (collection)
      2022-07-14 16:15:00 (collection)
      ...

Because of the way the API was designed, each of the time series data points are sub-collections, with a key that is data. The API would have been a lot easier to consume had the time series been expressed as an array, but this experiment has given me a good chance to learn Make and exercise its flexibility :smiley:.

I understand I could let my module output as-is and leave it to the user to use iterators, repeaters, etc. to process the time series data downstream, but this would generate many operations so what I’m after is for my custom module to generate output in this structure:

Bundle 1 (collection)
   Meta Data (collection)
   Time Series (*ARRAY*)
      Intraday (collection)
      Intraday (collection)
      Intraday (collection)
      ...

Each Intraday would look like this for example:

Intraday (collection)
   Date and Time: 2022-07-14 18:45:00,   // note the key is transformed to data
   Open: 139.24,
   High: 139.25,
   Low: 139.24,
   Close: 139.25,
   Volume: 259

I feel like the answer (if this is possible) is somewhere in Interface and/or Communication > Output, maybe including use of toArray, but I can’t seem to hit on it.

Thank you for any advice or direction!
John