How to limit parallel executions / create a custom queue?

[Jarno]

I have an Airtable database which reseives data via a form. When new record is added via the form, an automation launches and sends data to a webhook in Make. Make scenario includes a DALL-E module which creates an image.

The issue here is that OpenAI API has rate limits (my current limit is 7 images per minute).

When I get a flood of inputs, I will encounter an error with DALL-E shutting the scenario down. This I managed with “resume” error handler with a placeholder image.

However, now that the scenario runs even if an error occurs, I get a lot of placeholder images.

How can I manage the queue so that the scenario does not run more than 7 times a minute? Is there some kind of limiter or waitlist feature which limits how often the scenario runs, e.g. 7 times a minute?

[alex.newpath]

Yes there is – it’s called the sleep module – you can put artificial pauses into your scenario to slow down how fast it runs every time you are running it. This module can be placed just about anywhere including just befure the resume error handler.

[Jarno]

Thanks! But if I get 200 parallel (or almost parallel) webhook runs, wouldn’t the sleep module just pause all with the same amount of time, causing the exact same issue just to postpone a bit?

[Benjamin_from_Make]

Hello,

@alex.newpath is right. However I think that using this will not completely fix the issue, because if you get a burst of calls, all scenarios will pause and will (more or less) finish the sleep the same time…

Make doesn’t provide a way to configure the webhook queues. however, here are a few options you can consider:

1. Scheduling the called Webhook Scenario

Here, you can see that by default, the scenario executes immediately when a new message is received by the Webhook

If you schedule your scenario, the associated webhook will still receive the requests, but they will be picked regularly.

And with the Scenario setting “Max number of cycles”, you can define in some ways “how many messages” you grab once. If you set it to a maximum acceptable for Dall-E (ex: 6 or 7), you should be fine.

BUT, you have to be careful of 2 things:

• A scheduled scenario consumes operations. If you set the scheduling to 1 min, it it consume at least one operation per minute; which can end-up using a lot per month (you can still fine tune scheduling to not do this the whole day for instance). I would suggest a longer time.

• The queue of the webhook will fill with all the requests. It’s limited (based on your Make plan); so you have to be careful that, if you don’t consume enough messages vs what you get, you may have the Webhook queue full. It can be checked here
  

  

So, you may have to fine-tune the scheduling and the “Max number of cycles” value.

2. Set your scenario to run sequentially and not in parallel
This can be achieved here:

This option will make sure that your scenarios execute one after the other, and not in parallel.
The advantage is that, if you combine it with a Sleep module, as suggested by @alex.newpath, you are sure to never reach the limits of Dall-E.
BUT, you have to be careful, because if you get many requests, your Webhook queue could become full.

3. You use a Error Handler and use Sleep, Dall-E and Resume
For this, you let Dall-E fail from time to time, and when it happens, you try and replay after a sleep.

Here is an example

The second Dall-E is a clone of the first one, so that he does the exact same thing, and the resume picks the result so that next steps retrieve them as if the initial module didn’t fail.
It can limit errors, but you may want to filter the error handler so that it’s executed only when you have quota errors.

4. Use a Break error handler
Error handler allow to store the current bundle execution in a queue, so that it continues execution after a certain number of minutes

You first need to “Allow Storing of Incomplete executions”

Here, I configured it to try 3 times, and to wait 2 minutes between retries.

Here is what monitoring look like (when it fails)

When it fails, it goes to “Incomplete executions”. It will replay automatically after n minutes. If you have set “Automatically Complete Execution” to NO in the Break directive, you will have to resume execution manually.

It’s a lot of things, but I believe that at least one can be helpful

Benjamin

[Jarno]

Thank you! I think we can manage the queue well with at least two of your suggestions. I will try with stacking dall-e modules with error handlers. This sounds like the best option to manage a sudden load of webhooks which need to be operated as soon as possible, but without interruptions.

Very helpful, thank you.