Difficulty in aggregating JSON outputs from OpenAI after processing WordPress post information

Alex_Rey · November 2, 2023, 10:48am

Hello everyone,

I’m reaching out to the community for some assistance with a challenge I’ve encountered while working with make.com in my workflow.

Goal:

My goal is to create a workflow that does the following:

Retrieves post information from WordPress (3 to 5 bundles).
Sends this information to OpenAI to summarize each post.
Receives the summarized output in JSON format from OpenAI.
Parses the JSON output for each post.
Aggregates these summaries into a single output.

The intention is to have a consolidated summary of multiple WordPress posts, which I plan to use for content analysis and reporting.

Steps Tried:

So far, I have managed to:

Successfully retrieve post information from WordPress.
Send the data to OpenAI and receive the summarized content in JSON for individual posts.
Parse the JSON output to extract the necessary information.

However, I am stuck at the aggregation stage. The main issue is that the workflow processes each bundle of data from start to finish individually, and I can’t seem to figure out how to combine the JSON outputs into one.

I have tried using the text aggregator available in make.com to merge the outputs but to no avail. The workflow seems to treat each bundle separately, which prevents me from combining the data as intended.

This is a sample of the output I get from the OpenAI module that I proceed then to parse with parse JSON module:

{
"Title": "Holiday Marketing 2023: Insights and Strategies for Success",
"Content": "Gain valuable insights into consumer attitudes and make informed decisions for a successful holiday season in 2023. Discover key predictions and trends, such as the shift towards online shopping, the optimal timing for holiday shopping, and the ongoing relevance of Black Friday. Understand consumer spending habits and budgeting trends that will impact your holiday marketing strategies. Download the full guide for a comprehensive overview and get ahead of the game this season."

}

My final goal would be to merge all these in a summaries in a single text, which would look something like this:

Title: Holiday Marketing 2023: Insights and Strategies for Success
Content: Gain valuable insights into consumer attitudes and make informed decisions for a successful holiday season in 2023. Discover key predictions and trends, such as the shift towards online shopping, the optimal timing for holiday shopping, and the ongoing relevance of Black Friday. Understand consumer spending habits and budgeting trends that will impact your holiday marketing strategies. Download the full guide for a comprehensive overview and get ahead of the game this season.

Alex_Rey · November 2, 2023, 2:35pm

Still looking into this, to clarify the issue in more detail, what I need basically is to process ALL bundles in OpenAI BEFORE moving further in the scenario, so when the scenario continues to the aggregator, ALL bundles are already in OpenAI so they can be aggregated. Otherwise, it doesn’t work.
Hope this makes sense

Philippe_Billet · November 2, 2023, 3:15pm

Hello Alex,

Please find in attachment a blueprint that could help you (at least I hope).

Difficulty_in_aggregating_JSON.json (5.9 KB)

If I well understand, your data looks like this:

Then using one line of the array, I generate the data structure of the JSON parser.

Last step: the text aggregator building the final text:

i hope i’m not off-topic …

PBI

Alex_Rey · November 20, 2023, 11:37pm

Hello Philippe
First of all sorry for the delayed answer, had some personal issues but im back to work.

Thanks for you help, unfortunately I could not fix it. But I think I was not very clear either.

The issue is that each bundle is a different opperation in OpenAI. So the root of my issue is aggregating multiple operations bundles.

I’ve created a simple blueprint to clearly illustrate this issue.

The functioning is very simple:

Parse JSON where there is a object named ‘books’ with an arrays of books titles
Iterate over the ‘books’ array, resulting in multiple bundles

image379×601 7.35 KB
Sends each bundle to OpenAI to generate a brief summary, this results in multiple operations, and each operation generates a bundle

image437×837 34.4 KB
Set variable: To simplify data, I set a variable named ‘Brief Summary’, getting the output generated by OpenAI, but again this results in multiple operations with multiple bundles.

image438×771 34.1 KB

Basically I am stuck at this point: I have been able of parsing the array of book titles, and iterating over each one of them, creating multiple summaries with OpenAI, which results in 4 operations in this casem each one containing 1 bundle. And that is the issue, if I obtained 4 bundles that would be easy work, is the fact that i get 4 operations with 1 bundle in each one that makes it hard.

Here is a screenshot of the overall scenario:

blueprint (2).json (16.4 KB)

I tried to explain it more clearly this time, I hope to have been able to accomplish it.

Philippe_Billet · November 21, 2023, 7:34am

Hi @Alex_Rey,

Thanks, it is really clear now.
I looked at some similar questions in this forum (with keywords like “group operations”, “merge operations” …), it seems that the way you’re trying to aggregate is not the good one. But currently, it is not clear for me too…

Could you try to use the text aggregator just after the OpenAI module (and suppress the “Set variable” and “Array aggregator” modules)?

Philippe

Alex_Rey · November 21, 2023, 3:50pm

I’ve tried with both Text Aggregator and Json Aggregator, but nothing, since it’s different operations it would aggregate the bundles of each operation separately, in this case, only 1 bundle.
So the result is 4 operations for the text Aggregator.

Seems like a fairly simple thing what I am trying to do, so It would be hard for me to believe that make don’t allow to merge different operations. There must be some way

Alex_Rey · November 21, 2023, 3:59pm

Okay, just figure it out!

The key to MERGE different operations bundles is:

So the key factor is WHICH source module is selected for the Agg.

If you want only ONE operation as output, you must select the source module that only has ONE operation as output.

This thread was of help: Combine Operation Values into single string - #13 by Logan_R

I selected as the source module the first JSON, which is 1 operation, and now OpenAI module process ALL BUNDLES BEFORE moving forward to the text aggregator, so when the text aggregator is triggered, it merges all the bundles OpenAI into ONE.

Michaela · November 22, 2023, 6:50am

Hello there @Alex_Rey

I’m just quickly stepping in to say awesome work figuring this one out

Also, thanks a lot for remembering to share your findings and your final setup with us. This is incredibly valuable and has the potential to help many others who are looking for similar info in the future

Keep up the great work

Simon_Langham · November 22, 2023, 8:49am

@Alex_Rey

That’s really useful info.
I hadn’t made that connection until you pointed it out (ok via another post but you still did the leg work yourself)

Nice work!

Cheers

Simon

Make_Bot · December 5, 2023, 4:07pm