Thousands of Duplicate Runs - Quota Overruns

Anyone know why it occurs or how to deal with the constant creation of duplicate actions in our sequences or duplicate actions causing $50 plans to turn into $500 plans really quickly?

Some screenshots:

Lately, when I open a seq I’ll look for some error or alert. Inevitably when I move that action it reveals a duplicate underneath. When I remove that it forces me to chase the duplicates up and down the path - every delete causing another duplicate to be created or made visible. Often I need to delete 20-30 steps of duplicates. Nothing unusual In the sequences.

Especially frustrating when I get a notice that Google Docs cut me off 1/2 way through the night - even though I’m updating a doc 1x every 5minutes! Also, I’m now paying at least $300-$500 and the app isn’t in production - due to operation overruns.


Same trouble, but no reaction from Make support.
It was much better when it was Integromate.

You’re kidding me. I keep thinking I’m mysteriously doing something… holding down some mystery key. Here is a video:

hmm, your Video indicates there is a serious bug in your flow, you should check the JSON of the “export scenario” to look for inconsistencies.

Separately,

I run between 30 Million - 100 Million new rows of new data from APIs and web scrappers through make: make and google sheets per month and have been able to optimize it down under 100k total operations per month with a 5-minute update interval, including ETL/data transformation and cleaning.

It’s been a few months since my last operations usage whoopsie, but almost always discover that it’s because I made an incorrect assumption about the data, or rushed through testing the flows. If operations are being miscalculated due to some issue with a specific scenario, that should be straightforward to identify in the logs and submit to support.

My Most Recent Operations Usage Mistake Story.

OK, I don’t know what on earth I’d be looking for but I’ll look. This is a scenario that occurs randomly - although a bug could be random too.

Happy to take a look if you are able to share the scenario.

Honestly, this one is gonna be hard to troubleshoot without direct access to the environment, if you are not able to share your best option will be to go through customer support.

Quick Tip!

Modularizing your scenario into multiple separately callable “reusable purpose scenarios”

Will make it a lot easier to troubleshoot, and generally less likely to have conflicting “puzzle pieces” and more maintainable as you add calculations/features/toggles going into the future.
image



It also has the added advantage of giving you a “filter step” for the IML formula editor, to not be flooded with unused variables at your last module. you can choose which data is passed down in the webhooks to keep the work environment clean/minimal.

Scenarios are Triggered by webhooks in HTTP modules in prior steps.

1 Like

One more quick Tip.

It’s probable you might be able to combine these “set variable” module groups highlighted.

into a single module for each group “set multiple variables” saving operations.

1 Like

These are really great and I appreciate it. I just received an email back from support and apparently if I have a search step that returns 3 bundles the entire seq is repeated for all. Seems crazy but explains a lot. I killed that but still have something causing the entire seq to repeat intermittently. Now I see logs show cycle 1, 2, etc. But no indication what’s triggering it. Since it restarts from the very first step I wonder if it has to be the first step which is a slack comment.

OC :slightly_smiling_face:

You can tame these by following searches with one of the aggregator family of modules (array aggregator/text aggregator/table aggregator/csv aggregator/etc)

or applying a filter that only allows the first/last/other criteria bundle through.

On the flip side, the way lists of items are handled is where a good chunk of the power of make for data transformation exists.

1 Like

I’m looking into combining variables, but seems steps need to be distinct to reference each other?

Yea, if there is a way to kill the unneeded bundles it’s powerful. I’ll look into those.

Correct,

Alternatively Though: you could also do a google sheets helper page pass through, per my response in the other topic here. to combine 6 sequential calculations into 2 operations.

I use this method to combine tens of thousands of calculations across whole datasets into two make operations.

There are a lot of strategies for optimizing operations use: but also want to be careful not to blindly recommend more optimization methods not knowing what formulas are used in each module and where each variable is called. It could lead you down unproductive rabbit holes/wasted time as you learn the platform

1 Like

Actually I’m using the google sheets search steps as a honeypot to trap the bot on additional passes - so I can probably do away with them completely eventually. But I’ve still got something trying to repeat the whole damn seq. I’ll try up upload seq here. actually, that might expose some endpoints that it’d be a pain if people hit.

It is weird the start step would commit at the end of the sq?
Here is a screenshot:

So the 2nd step in sq is to split the 1st step message into 2 parts. You don’t think that would spawn additional sq instances, do you?


2 Likes