Operational Efficiency

[Grahame_Bull]

Hi, i am brand new to Make and so far it looks fairly useful for backend operations. My main use for Make would be to schedule retrieval of data from 3rd party api and then to store that in my Supabase table.

However im struggling to find the efficiencies due to the operation cost per insertion of row.

I have a scenario where i have 4 steps.

1. Get HTTP request
2. Parse JSON
3. Iterator step
4. Update Postgres table (Supabase)

Lets say the JSON returned has c400 items. The way i have the scenario set up would cost me 400 operations. The kind of data i would be moving could go up to, and beyond 10,000 per week. Im on the core plan. (which i am using to test make’s viability for my use case) and i dont think that would cut it.

Is there a way to reduce the operations needed to insert rows into a table. Ive read somewhere there are batch insert modules but i cannot find them?

Any help from anyone whos had experience with this type of scenario before would be great. If it is indeed 1 operation per row then its probably best to ditch Make as a viable option and just code it

[Runcorn]

HI @Grahame_Bull,

I need to connect with PostgreSQL to test it and see if it is possible to do so or not, But, Can you check this for me,

Execute a query (advanced): Can you see if it allows you to write multiple update statements? If so, Can you check on how much it supports query length/number-wise?

You don’t need to use the ParseJSON but I guess removing it will only save a single operation

If it supports multiple update statements, we can basically construct the Update Statement in Bulk using Iterator and Text Aggregator which will essentially drop the cost from 10k operations to 2, if there is a limit then you might need to do it with a fixed number of queries.

Having said this, You will need to review it and see if it is possible at all or not.

[Grahame_Bull]

Hi - thanks for the reply.

Where would i be looking for Execute a query (advanced):? in the postgres module?

in the module (insert a row) there are only the options shown below

[Grahame_Bull]

Sorry i found the Execute a Query module in postgress