How to read a relational database and save as table

Hello Everyone,

I want to read data from a relational database and save them in a table weekly. I almost went through all steps but my scenario consumes operations massively.

I use HTTP requests to read my data from an API, the response format from the API is like this:

[{“id”:1001,“name”:“Name01”},{“id”:1002,“name”:“Name02”}]

Each ID has lots of attributes, I can also read them using separate requests and a sample response will be like this.

[{“id”:1001,“status”:“Status01”},{“id”:1002,“status”:“Status02”}]

I need to make a table like the below one from this data:
image

My scenario starts with an HTTP request, then I parse the response using parse JSON and then use add the record to the data store or update the record in the data store, to save this data in the format I want.

The problem is, make.com runs 1 operation for each bundle to add a record to the data store (In this scenario I will have 1 bundle for each record). I have around 10,000 records in each Project and up to 20 attributes for each record. With this method, the number of operations will be massive and it is not logical. Does anyone know how can I reduce the operation consumption in this scenario? Or as I am not sure if I am on a right track, is there any other solution for my main problem (read merge and save the data)?

You can also find the module’s setups below. They may be helpful to understand the scenario better.

Thank you,
Mostafa


Hi. Unfortunately there is no way for you to load bulk in the datastore.

If you need additional support, please don’t hesitate to reach out.

Cheers
//Helio