1. The Challenge: Hidden cost
Before optimisation, our scenarios were incurring significant, unnecessary costs due to frequent and redundant file lookups.
-
Files: We rely on two critical files that are generated and saved monthly in a folder structure organised by year->month->client.
-
Earlier: Every time a scenario needed to access these files, it had to execute a sequence of multi-step searches to locate their unique file IDs.
-
Cost per run: 2 file searches, causing 3 operations/search = 6 operations/run.
-
Monthly cost: 20 scenarios, around 500 runs/scenario = 60,000 operations/month just for file lookup!
-
The catch: The file IDs only change once per month, yet we were performing the lookup 60,000 times.
2. The Solution: Centralized scenarios for updating the database file IDs using Custom Variables
To solve this, we used Make’s built-in tools, the “update custom variables” module, specifically in this case.
Step 1: The Central Updater Scenario
We created a single, scheduled scenario that runs twice per month, on the date the new reports are expected to be generated, and one run just in case the files are not created by the previous date. This small scenario takes just 9 operations per run, which means just 200 operations per year.
Scenario:
1. This scenario does the necessary searches (Year → Month → Client Folder → DB1) and similarly for DB2.
2. It uses the “Update a Custom Variable” module to store the two newly fetched file IDs into two dedicated organisation-level variables:
-
DB1
-
DB2
Step 2: Using these custom variables in individual scenarios
In the 20 scenarios that need these files, the search modules were removed, and instead, wherever those file links are needed, We added the module links directly from the custom variables.
Cost Impact: Apart from the time that was used by scenarios to cover those additional 6 steps, the cost impact was huge.
The 3 operations per file search (6 ops total) are now replaced by 0 additional operations, as accessing a custom variable is an internal process that doesn’t take additional operations.
Earlier, these 20 scenarios were taking 60,000 operations per month (7,20,000 operations per year), and now, these are replaced by an annual 200 operations for the central scenarios.
The learning - the basis of automation is to reduce the repetitive work of humans that have similar patterns then why not use the same automation to reduce repetitive work of the other automations.
this is a killer feature! You can use your entire years’ of credits anytime within the billing cycle, instead of expiring credits each month!