Summary of improvements for scenarios interoperability

[ProLingua]

Improvement requests keep popping out about the HTTP module to be able to better communicate between scenarios; the main problem is it stopping after a short time or not being able to handle chunked requests or pagination. We often have to use them to make scenarios operate together, because the solutions offered by Make’s support team (using a Data store or calling other scenarios through the Make or Scenarios apps) are not enough. For this reason I try to summarize these App Improvement Ideas and the posts in the community: we are in dire need of upvotes to these App Improvement Ideas.
If you agree, please upvote the main request for every topic.

Longer timeout times
This would help manage scenario interoperability as well as API calls to other services which need longer times (such as AIs):

• This is the main request, which would be meaningful to upvote: https://www.make.com/en/app-improvement-ideas/p/http-longer-timeouts-on-http-module
• A smaller problem which didn’t take into accoutn the possibility to raise the limit from 40 to 300, but 300 is still not enough: https://www.make.com/en/app-improvement-ideas/p/webhooks-webhook-response-incoming-trigger-http-webhook-specify-timeout-value-la
• Same problem, here they suggest using Data Stores, but we know it isn’t enough for complex systems: Timeout request - can it be increased? - #4 by ManishMandot
• This post talks about using HTTP modules and webhooks, but we know the timeout is going to be an issue: Getting scenario output with api - #5 by samliew
• Here is an unhelpful response about timeout limits: HTTP request runs into timeout - #6 by samliew
• Post about the maximum 300 seconds timeout: Timeout warning in a http request
• Here people suggest closing the HTTP request early and ask later for the output, which sounds like the Data Store solution, but it brings problems like the size of the payload to store, its structure to be stored inside the Data Store, knowing when an operation has finished and has written in the Data Store: Module timed out - #2 by ManishMandot
• In this post people discuss about asynchronous calls and how to catch the output of a scenario which could take too long of a time to be called with an HTTP module: Asynchronous HTTP call, how does it work in Make? - #7 by Henk-Operative
• The response in this post is directly negative: Handling http requests that take longer to respond than 300ms
• Here the poster just surrended: "Fire and forget" HTTP Call - #2 by samliew
• Here people suggest splitting big scenarios with HTTP requests, which is our common solution to create reusable scenarios and bypass the 45 minutes limit of a scenario, but the problem is the 300 second time limit for HTTP modules: Scenario execution delay

Pagination integration
This would help manage scenario interoperability, but we still lack the ability to send chunked responses to webhooks; nonetheless it would help with lots of API integrations:

• Main request: https://www.make.com/en/app-improvement-ideas/p/http-integrated-paging-of-api-results
• Additional request: https://www.make.com/en/app-improvement-ideas/p/http-pagination-support
• Post about optimizing HTTP calls to Make Webhooks bypassing limits: Handle HTTP timeout - #3 by AV_SundaeLane

Chunked webhook responses
This would help manage scenario interoperability, as it would help bypass the timeout limit of the HTTP app and would help us create scenarios able to paginate their content.

• Main request: https://www.make.com/en/app-improvement-ideas/p/webhooks-chunked-responses

Scenario outputs
We have both a “Make - Run a Scenario” and a “Scenarios - Run a Scenario” modules, but none of them returns the output of the called scenario. This means we can call different scenarios but we cannot get their outputs; again, Make’s support team solution are not enough, as Data Stores are not the best way to make scenarios interoperate. To me it seems pretty obvious that we need the possibility to get the output from a subscenario.

• Main request, for the new “Scenarios - Run a Scenario” module: https://www.make.com/en/app-improvement-ideas/p/run-scenario-and-receive-output
• Lesser request, for the old “Make - Run a Scenario” module: https://www.make.com/en/app-improvement-ideas/p/run-a-scenario-and-receive-output
• Here the Subscenario feature is referenced but never expanded upon, and we know we cannot actually get outputs from subscenarios: Run a scenario from Make.com Parent Scenario
• Same problem: Pass value from one scenario to another - #2 by Runcorn
• Same problem, here the last post talks about posting outputs from the “Run a scenario” module, which in reality does not work: Use existing scenario output in new scenario - #2 by AnthonyatXRay

[Henk-Operative]

Hi @ProLingua,

Thanks for taking the time to set this out. I see many good improvements and I agree with this.

Have you tried building your own HTTP module that meets your requirements, using the Custom App Development documentation?

You still can’t bypass Make’s timeout with this, but it gives you free control in handling pagination and page sizes and rate limits. (timeout is even worse at 40 seconds, I realize now)

For requests where it simply takes too long for a response to be returned (often with AI models), I use Xano as a proxy:

1. HTTP in Make to Xano endpoint to start the flow (including a Webhook URL from a separate scenario).
2. Xano calls the external service, waits for the response
3. Xano formats the response and sends it to the given Webhook URL
4. Process the full response in Make as I wish.

In combination with a Custom App, I can quickly set up a proxy call with Xano with whatever service I want. But you should know that I am really into building advances stuff, this is not what regular Make users should have to do.

Cheers,
Henk

[ProLingua]

Building my own implementation of the HTTP module was my next attempt: I realize that I rely so much on scenarios talking to each other that at some point I will need to do it myself.

Thanks for the proxy idea: I was thinking how I could create a system like this in Make; my main problem is that I often use a master scenario which calls a serie of subscenarios, and switching to an entirely different design requires time. I will certainly take a look at Xano (which seems like a more advanced and costly version of Make), but maybe I will be able to create it with Make scenarios or a custom app.

I do not want to steal your work, but do you have any suggestion about the custom app development?

[Henk-Operative]

I use both Make and Xano and love each tool very much. Make is the best choice for automation, but lesser for a backend. (You would NOT want to run a database for your web application in a Data Store lol) Xano is as advanced as you make it; you can do highly complex stuff in the function stack, including Lambda functions. It really is a great tool to take care of your whole backend; database, server and API’s. It is amazing what it will do, without any code.

Here is a 10% off affiliate link for Xano: Xano

As for the custom app, if you don’t have experience, it is a good start to follow the custom app development badge on the Partner Training platform. There is also the custom app development documentation.

• Specialty Badge - Custom Apps Development Training
• Make.com - Custom App Documentation

The way I implemented it is that I have a Custom App for ‘Operative’ (my company), it has several tools/modules that I use on a daily basis. One of the modules is Xano. In Xano I have some prebuilt endpoint, one for each external service. In the module in Make, I can select which service I want to use, which connection, the response webhook URL and the payload itself. The module takes care of the rest.

Cheers,
Henk

[ProLingua]

That’s an amazing work you are doing. Thank you very much for the links.
I’ll leave this post open and not check your answers as solution because the post is still about Make improvements.