How to create batch size of 100 rows and send to HTTP Make a Request which will accept JSON body

:bullseye: What is your goal?

Google Sheet => HTTP Make a request for Supabase Insert/Update in table named ‘property’ (UPSERT operation)

Google sheet has 5000 rows with 3 columns like external_id, name, city
Supabase : table ‘property’ with column external_id, name, city
created UNIQUE Constraint on external_id
Enabled RLS . Also created all the policy for Select , Insert, Update

:thinking: What is the problem?

Currently am testing in free plan.
I am using Google Sheets => Array Aggregator => HTTP Make a Request (using API Key)

But not sure how to prepare the batch size of 100 rows, so that i can 100 rows at a time to HTTP to minimize the API call . Also tackle the timeout issue and performance

:test_tube: What have you tried so far?

Google Sheet => Array Aggregator => HTTP Make a Request

Hello,

It depends on how your Supabase endpoint is implemented.

If your current setup accepts only one record per request, then you won’t be able to avoid making multiple requests. In that case, batching on the Make side does not make much sense.

However, Supabase can accept arrays for insert/upsert, so you can send multiple records in a single request.

What you can do in that case:

  • Use Aggregate to JSON to build a proper array payload, or use Text Aggregator if you have simple data and don’t want to work with data structures
  • Send this array in a single HTTP request to Supabase

This way, you reduce the number of operations and avoid timeout issues. But as mentioned - if the endpoint accepts only one record per request, you won’t be able to avoid making multiple requests.

Hope it helps!

Have a nice day,
Michal

Thanks Michal. Aggregate to JSON worked. But in the internet I can see many people says, in Production, Instead of 5000 rows → 1 request , Do 5000 rows → chunks of 100 → 50 requests. How to do this ?

Welcome to the Make community!

If this is what your scenario looks like, then you currently have an array from the aggregator module.

I have a chunk array module that you can use.

Here’s how to use it, if you are looking to process 100 items in each bundle:

My Custom AppAbsolutely Free!

Save headaches and operation credits with these utility modules:

  • Chunk Text
  • Chunk Array
  • Chunk Words
  • Multiple Find & Replace
  • Collection to Array/String List
  • Execute JavaScript
  • Estimate Tokens
  • Calculate Difference Between Two Dates
  • Get Next Business Day (with holidays)
  • List Upcoming Dates of Day of Week
    and more!

Hope this helps! If you are still having trouble, please provide more details.

@samliew

Thanks. I Will check.