What are you trying to achieve?
Hi Make.com Community,
I’m working on automating a Python script running in Deepnote when a webhook is triggered in Notion. Here’s the basic flow:
- Webhook is triggered by Notion.
- The ID is sent to Make.com, where it’s saved using the “Set Variable” module. (Note: This needs to be sent to Deepnote, but Deepnote doesn’t accept variables as part of a POST request, which is why I’m considering using a Google Sheets log instead.)
- A subsequent HTTP POST request is made to trigger my Deepnote notebook, where my Python script executes.
The python script works fine for one request or multiple requests spaced out enough. However, I’m concerned about handling multiple webhook requests that might arrive while the Python script is still running. The python script can take anywhere from 20 minutes to 20 hours, depending on the data.
While I’ve enabled sequential processing, there’s currently no way (I know of) to notify Make.com when my python script has finished running. Without feedback to indicate when the process is complete, sequential processing won’t fully solve the problem.
The main issues I’m facing and some potential solutions I have thought of, but not successfully implemented:
-
HTTP Get Request: I considered having Deepnote send a “Done” status back to Make.com via an HTTP Get request. However, Make.com’s scenario timeout of 45 minutes is too short, especially when the python script might take longer than that. Is it possible to extend this time or handle it differently?
-
Using Google Sheets for Logging: Another approach I’ve considered is logging the request IDs in a Google Sheet. This also solves the problem of Deepnote not accepting direct variables sent to it via the trigger POST Request. The Deepnote script could then process the IDs and update the status in the sheet. However, my concern is that Deepnote might overwrite Make.com’s logs when writing back to the sheet. Additionally, I’m unsure how to ensure the “Done” status is updated without causing conflicts, or how to use Make.com’s Search Rows module to check when a task is complete without encountering the 45-minute timeout issue.
-
Multiple Notebooks for Concurrent Requests: I’m also exploring the idea of using multiple Deepnote notebooks to handle requests in parallel. For example, I could use 5 notebooks, where:
- The 1st request goes to notebook 1.
- The 2nd request goes to notebook 2.
- The 6th request loops back to notebook 1, with the hope that it’s done processing the 1st request by then.
Is this feasible in Make.com? Can I set up Make.com to loop through available notebooks and send a request to a different notebook each time a webhook is triggered?
-
Multiple Scenarios Working Together: Another potential solution is setting up multiple scenarios to handle parallel requests. However, I’m again unsure how to notify Make.com when the Python script in Deepnote has finished processing so the scenario can complete properly.
What I Need:
Ideally, I want a system where multiple requests are handled one at a time. Each request should trigger the Python script in Deepnote, and only after that script finishes, should the next request be processed. This would ensure no data is lost, and everything runs smoothly without conflicting updates.
I’m relatively new to Make.com (about a week in), and I’d greatly appreciate any advice, suggestions, or guidance on how to best handle this situation.
p.s. python script/Deepnote script/script all refer to the same python notebook running in Deepnote.
Steps taken so far
Setup a scenario which accepts custom webhook, saves a variable (id), sends the id to google sheets, triggers execution of python script using HTTP Post request
