Retrieving Iterator bundle data after Text Aggregator

[Bob_Balm]

Hi,

I have the following setup:

Basically, these are chat conversations from a chatbot application. For each conversation, I’m combining all the messages into one text variable (the Iterator and the Text Aggregator), so that I can feed all of that to OpenAI to have it determine the topic of the chat. So far so good, this works swimmingly.

But now I want to put the result into a Google Sheet, with columns being:

• Conversation ID
• Conversation Date
• User (email, if provided)
• Topic
• Chat log

The topic (result of Open AI module) and Chat log (result of Text aggregator) are available just fine, but I can’t reach the Iterator data from the Google Sheets module, to get the other data of each conversation. I also tried getting it from the JSON parser, but then it just gives me the data for the first conversation on all of them. These are my options on the Google Sheets module:

This intuitively feels like something that shouldn’t be too difficult to set up. But what am I missing?

[samliew]

@Bob_Balm

I also tried getting it from the JSON parser, but then it just gives me the data for the first conversation on all of them

That’s how arrays work. You need to specify the index of the array, or iterate through each item (like using the iterator).

If you need further assistance, please provide the Output Bundle of the Parse JSON module.

Please provide the output bundles of the modules by running the scenario, then click the white speech bubble on the top-right of each module, save the bundle contents in your text editor as a bundle.txt file, and upload it here into this discussion thread.

Providing the output bundles will allow others to replicate what is going on in the scenario even if they do not use the external service.

This will allow others to better assist you. Thanks!

[Bob_Balm]

So, this is what that looks like. I’ve redacted the content and limited it to two bundles, but it’s basically 100+ of these puppies:

[
    {
        "data": [
            {
                "id": "v-_Be3t28SkwhYibpDgKE-7El4_A1HsJe3Z9ru7ASCy",
                "created_at": "2024-01-01T17:46:10.243566+00:00",
                "messages": [
                    {
                        "content": "redacted",
                        "role": "assistant"
                    },
                    {
                        "content": "redacted",
                        "role": "user"
                    },
                    {
                        "content": "redacted",
                        "role": "assistant",
                        "score": 0.884679274735721
                    }
                ],
                "chatbot_id": "7El4_A1HsJe3Z9ru7ASCy",
                "customer": {
                    "name": "redacted",
                    "email": "redacted",
                    "phone": "redacted"
                },
                "source": "Widget or Iframe",
                "min_score": 0.884679274735721,
                "country": "redacted"
            },
            {
                "id": "zbAPJJlSOkc4YA9TXQ017-7El4_A1HsJe3Z9ru7ASCy",
                "created_at": "2024-01-01T15:31:03.68745+00:00",
                "messages": [
                    {
                        "content": "redacted",
                        "role": "assistant"
                    },
                    {
                        "content": "redacted",
                        "role": "user"
                    },
                    {
                        "content": "redacted",
                        "role": "assistant",
                        "score": 0.873426409934701
                    },
                    {
                        "content": "redacted",
                        "role": "user"
                    },
                    {
                        "content": "redacted",
                        "role": "assistant",
                        "score": 0.7540962312388952
                    },
                    {
                        "content": "redacted",
                        "role": "user"
                    },
                    {
                        "content": "redacted",
                        "role": "assistant",
                        "score": 0.7782639542344136
                    }
                ],
                "chatbot_id": "7El4_A1HsJe3Z9ru7ASCy",
                "customer": null,
                "source": "Widget or Iframe",
                "min_score": 0.754096231238895,
                "country": "redacted"
           }
        ]
    }
]

This output goes through the Iterator, so each bundle (conversation) is iterated through. The text aggregator creates one text variable of each conversation’s messages array’s content properties. From there it works fine, but I need to retrieve the id, created_at and customer - email properties as well, and it looks like the Text Aggregator stops me from accessing that information after that module.

[samliew]

Okay, I can see now that there are two levels of arrays involved.

So you’ll need to have two Iterators, one for conversations, and one for messages in each conversation

This way you still can access the conversation variables even after aggregating the messages.

[Bob_Balm]

Hmm. While that does let me select the variables from the conversation, it’s still only loading the first conversation’s details.

What strikes me as strange is that it has only 1 operation for everything up until after the Text aggregator. Only the OpenAI and Google Sheets modules rack up Operations while running, for each Bundle created by the Text Aggregator. I’d expect the iterators to cost 1 operation per iteration as well but it just doesn’t look like it’s iterating. But even so, I’m still getting neatly aggregated bundles from the Text Aggregator.

So if this is the output of the iterator:

[
            {
                "id": "v-_Be3t28SkwhYibpDgKE-7El4_A1HsJe3Z9ru7ASCy",
                "created_at": "2024-01-01T17:46:10.243566+00:00",
                "messages": [
                    {
                        "content": "This",
                        "role": "assistant"
                    },
                    {
                        "content": "is",
                        "role": "user"
                    },
                    {
                        "content": "text",
                        "role": "assistant",
                        "score": 0.884679274735721
                    }
                ],
                "chatbot_id": "7El4_A1HsJe3Z9ru7ASCy",
                "customer": {
                    "name": "redacted",
                    "email": "redacted",
                    "phone": "redacted"
                },
                "source": "Widget or Iframe",
                "min_score": 0.884679274735721,
                "country": "redacted"
            },
            {
                "id": "zbAPJJlSOkc4YA9TXQ017-7El4_A1HsJe3Z9ru7ASCy",
                "created_at": "2024-01-01T15:31:03.68745+00:00",
                "messages": [
                    {
                        "content": "So",
                        "role": "assistant"
                    },
                    {
                        "content": "is",
                        "role": "user"
                    },
                    {
                        "content": "this",
                        "role": "assistant",
                        "score": 0.873426409934701
                    }
                ],
                "chatbot_id": "7El4_A1HsJe3Z9ru7ASCy",
                "customer": null,
                "source": "Widget or Iframe",
                "min_score": 0.754096231238895,
                "country": "redacted"
           }
        ]

Then this is corresponding example output of the Text Aggregator:

[
    {
        "text": "This is text",
        "__IMTKEY__": "Yj9lolH5fAhPMRzAcd_Er-7El4_A1HsJe3Z9ru7ASCy"
    },
    {
        "text": "So is this",
        "__IMTKEY__": "BNDjMPzvm6AIgJlHao-7w-7El4_A1HsJe3Z9ru7ASCy"
    }    
]

And then for each of those it’ll run the OpenAI module just fine. But even with an extra iterator in front, I can now select the details in the Sheets module just fine, but it’s still only outputting the details of the first conversation on every row:

Results in Google Sheets: