How to Split data into batches and then use them to get output?

[pulkitd]

So, I have a notion database where each row has a column with a list of 31-41 ideas. It’s like Row 1 (some fixed text), column 1 (list of 31 ideas); Row 2(some fixed text), column 2(list of 33 ideas) and so on…

Now, I want to create an outline for each idea from this list of 31 or 41 ideas using Chatgpt. But I don’t want to feed them together and get an output for all 31 of them in one go. I want to batch-process them in a group of 5. So, for a row that has let’s say 32 ideas - they will be processed in 6 batches of 5 ideas and 1 batch of 2 ideas using chatgpt (one batch at a time, prompt will remain same).

And output for each batch should get consolidated in a new google document one after another and URL of that google doc to a new column in the respective row of notion database.

I tried many things (taking help of AI as well as google) but still not able to create and execute this scenario with success.

For context, how my notion DB looks like

Output from module I’m using (it updates the ideas generated in previous step using chatgpt)

[Benjamin_from_Make]

Hello!

If I understood, you get a Long string that contains the ideas separated by a new line, and headed by a number and a point, right?

By chance, is there a way you could have the ideas as separate elements in an Array? If not, if it can only be this Long string, one way would be to:

• use a Replace module (or function - but less convenient), and replace all “xx.” with a comma for example.
• split the resulting Long string with comma (you get an array).
• add a repeater where the “repeat” is calculated based on length or your array divided by 5 (and using the floor function to round)
• in ChatGPT, when you ask it to process each batch, use the Slice function (and join) leveraging the ‘i’ variable the repeater increments for you.

I attach an example blueprint. I did a longer version so that you see the details, and an optimized version that limits operations.

SliceIdeas.json (19.5 KB)

I hope it will help.

Benjamin

[pulkitd]

Thanks for the reply!

Yes, you got that right. To get the ideas as separate elements, I was thinking to create different sheets for a list of idea but then I guess it would complicate things.

The solution you gave worked but the issue is it stops in 3 repeats. The list has 31 items but repeater makes 3 bundles of 5 and stops. For some reason it’s not considering length as 31.

output from set variable module

output from repeater module

[Bjorn.drivn]

@pulkitd As described by Benjamin you need to split your text and make it an array. Right now your text is still the data type text, so the length() won’t work.

I created my own variation of this, maybe that helps you out. In this scenario I don’t use the RegEx but just split the text based on the newline between them. Also using a little bit of a different formula after the repeater (why make it simple when you can make it look hard

1. Make sure you split the text into items within an array. You can do this with the split() function.
2. Once the array is created you can use the length() function to get the length of the array.
3. Once you have the lenght, use this to initiate the amount of repeats you need to complete
4. Finally you need to grab items from your array with a formula that calculates this. Since these are items from an array, you need to use the join() again

You can replace the “Set variables” in the beginnen, I used that to create the text & array while testing.
Splitting text into batches and processing it.json (9.4 KB)

[pulkitd]

Thanks @Bjorn.drivn It worked seamlessly. Also, how can I consolidate this data generated by Chatgpt to update back to notion? I tried array aggregator and text aggregator but seems like I’m doing something wrong as it’s not working

[Bjorn.drivn]

Yes @pulkitd you could aggregate from the repeater. Make sure you set the repeater as the origin

[pulkitd]

oh okay! I was using chatgpt module as source. Using repeater as source solved the issue. Thanks a ton! @Bjorn.drivn

[pulkitd]

@Bjorn.drivn one last question. Is it possible to divide data into bundles of 4 from a list of 30-40 items like this

Bundle 1:

1. header 1
   description 1

2. header 2
   description 2

3. header 3
   description 3

4. header 4
   description 4

and so on. Basically, dividing the output in text aggregator that I got from chatgpt for further processing?

[Bjorn.drivn]

Yes you could do that @pulkitd . If you ask GPT to return it in a certain format, you could use this format again to split() and join() it however you like.