Hello all,
I am trying to upload a batch if pdfs to run an analysis with one of the OpenAI assistants we have created.
Each Clickup source I pick will have a variable number of files, so I cannot specific exactly how many I will have.
I have created a filter to wait for all operations in the iterator to complete, but when I run it, it only gives one vector array to OpenAI, instead of giving all files/arrays.
Any idea?
Welcome to the Make community!
Every result (item/record) from iterator/list/search/match modules will output a bundle. This can result in multiple bundles, which then trigger multiple operations in future modules (one operation per bundle). To “combine” multiple bundles into a single variable, you’ll need to use an aggregator of some sort.
Aggregators are modules that accumulate multiple bundles into one single bundle. An example of a commonly-used aggregator module is the Array aggregator module. The next popular aggregator is the Text Aggregator which is very flexible and can apply to many use-cases like building of JSON, CSV, HTML.
There are other types of aggregator modules, click the below links to find out more:
The Array Aggregator module is very powerful because it allows you to build a complex array of collections for a later module’s field to map multiple items (collections) to it.
This is done using the “Target structure type” of an Array Aggregator module.
Here is an example:
As you can see, the “Map” toggle on complex fields are used when you have an array. You can easily build an array variable to map to a future module’s field, by using an Array Aggregator module and select the “Target Structure Type” as the future module’s field you have mapped the array into.
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the Make Academy. Investing some effort into it will save you lots of time and frustration using Make later!
@Ludo What is the output of your Assistant module?
You say it gives one vector array, what are you expecting? Please provide some more information and output screenshots
Here is the output. Its basically only analysing the last file when I need it to run the analysis across all files.
My goal would be to download and upload all the files to the same vector file then just reference that one to chatGPT… but I need to create a new vector file every time if not it wont work.
No idea how to do this… I’ve been playing with array aggregator but it doesn’t seem to do the job as it doesn’t allow me to select multiple operations.
This is an example of what happens when I try to upload all files before moving them to the vector file. It only moves the file from the last operation instead of all files.
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the Make Academy. Investing some effort into it will save you lots of time and frustration using Make later!
Hey @samliew thanks for helping.
For some reason it’s still not working and only passes on the last operations output. I suspect the issue is with the array aggregator. See below.
Please note that I have a filter after the array aggregator to make sure it “waits for all files” so only when all the available payloads from the iterator are uploaded it will move to that step.
What is the Source Module of your aggregator? Is it set to the iterator?
I couldn’t tell from your screenshots.
No it was set to the upload a file module as I want to aggregate the files to upload into the vector storage as per below…
But I think your comment solved it!!! … I just had to set it to the Iterator instead and it can still use the Filed ID output.
Let me do some testing.
Summary for anyone reading this thread:
That’s a very common beginner’s mistake.
Always set the aggregator to where the multiple bundles are being generated. More often than not, this would be your iterator module.
Here it is again for your reading:
Every result (item/record) from an iterator/list/search/match module will output a bundle. This can result in multiple bundles, which then trigger multiple operations in future modules (one operation per bundle). To “combine” multiple bundles into a single variable, you’ll need to use an aggregator of some sort.
Aggregators are modules that accumulate multiple bundles into one single bundle. An example of a commonly-used aggregator module is the Array aggregator module. The next popular aggregator is the Text Aggregator which is very flexible and can apply to many use-cases like building of JSON, CSV, HTML.
There are other types of aggregator modules, click the below links to find out more:
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the Make Academy. Investing some effort into it will save you lots of time and frustration using Make later!
