Seeking Best Approach for Scheduled Automation in make.com scenario

How To
, ,
1

Hey Make community, I need some guidance on optimizing my automation workflow.

I have a Google Sheet with approximately 6,000 rows, and I’m using OpenAI and Perplexity to update data on that sheet. Currently, I’m running automation that processes 250 rows per run, but it seems to get stuck after processing 250 rows straight.

Ideally, I want the automation to:

  • Run every hour or half an hour, processing 250 rows each time.
  • Pause for 30-60 minutes after each batch to allow the system or APIs to reset/cool down.
  • Resume from where it left off, picking up from the last row it processed.

I’ve tried configuring the scheduling intervals in Make.com, but it doesn’t seem to align with what I need.

What would be the best way to achieve this? Is there a more reliable method or workaround to ensure the automation runs smoothly without getting stuck?

Any suggestions or best practices would be greatly appreciated!

Thanks in advance!

2

Hi @Arvin_Zatulovsky,

I think you could also split up the scenario into two scenarios.

  1. Scenario you search ‘all rows’ (obviously in the beginning start with a few to test it..)
    Then you use the HTTP module to start the second scenario.
  • you might update the rows as “triggered” or something to indicate that you don’t process the row multiple times.
  1. The second scenario starts with a webhook module. The URL you receive is the one you use in the HTTP module of your first scenario. You either pass a row number or the needed data from the spreadsheet from the HTTP module to the webhook and then do the OpenAI & Perplexity stuff…
    Make sure you have the setting “Sequentially processing” setup so you only do one row at a time. This way you can put all 6000 rows into a queue, go to sleep and wake up with all rows being processed.

Hope it helps!

Best,
Richard

2 Likes
3

Welcome to the Make community!

:up_arrow: This is the most optimal way if you need to use a queue for any reason in Make. I would have suggested the same method too.

To split data into chunks, see A Better Solution for "How to group bundles in X amount of bundles" (Chunking Arrays). This can be used to send data to the webhook queue, so that you do not need to perform another “search” in the second scenario.

Hope this helps! Let me know if there are any further questions or issues. P.S.: investing some effort into the tutorials in the Make Academy will save you lots of time and frustration using Make!

— @samliew