Hi everyone,
I’m working on a scenario that retrieves data from the Close API and generates multiple CSV files using the “CSV Aggregator” module. My goal is to combine all these CSV files into a single file so I only need to upload one file to Google Cloud Storage.
Currently, my scenario:
- Fetches data from the Close API using paginated requests.
- Aggregates data into CSV format.
- Uploads each CSV file individually to Google Cloud Storage.
Problem:
Instead of uploading multiple CSV files, I need to merge all the data into one CSV file and upload it as a single file to Google Cloud Storage.
Questions:
- What is the best way to merge all CSV outputs into one file?
- Should I use an aggregator, a data storage module, or an iterator loop to handle this?
- Are there examples or blueprints that demonstrate combining CSV files before uploading?
Any suggestions, tips, or best practices would be greatly appreciated!
Thanks in advance for your help!