How to pause a scenario and run in chunks

How To
,
1

Hello all,

I have Make connected with Airtable. I have scenario that first Collects data from Airtable, pick first 5 records, send that to HTTP request, which ill query data against those 5 records, parse data, and Update airtable with all fields that are mapped in “Update airtable”.
All this works in manual way with few problems that I will ask later in post.
Problem is that I need to automate this somehow but I have few limitations. I am on free plan both on Make and Airtable until I figure out what can work for me, and evaluate all this, which means I cannot use any advanced functions on both platforms.
Problem:

My HTTP request is limited to 5 records only within 5 mins, and i need to query against 100 records that are in Airtable. Timing is not a matter to me so this can all finish in like few days as long as it gets all data. The problem is that i don’t know how to make “Pause” and cut data from Make in chunks. There is "Pause function itself but that one is allowing only 10min query or it will fail, so this function is def not for my purpose. II cannot create multiple scenarios, that sound pointless, and this will grow (number of records to query against), and as last first Function is to query 5 records, which means every time it will query first 5, and I need to continue with next 5, and next 5 until reach end of table.

My scenario and my need is this:
What I need to do, send HTTP request, that will run a query based/against the first 5 records in airtable, collect data, Update in airtable, pause 5,10,20min it does not matter it can be more, then run HTTP request for next 5 records from airtable and so on until it finish. After that it should ideally stop as I don’t need loop. I need all this to be done once per month.

Problem I encountered with this scenario is also “Update airtable” part.

My table contains different fields some are $, some are Numbers format, some are timestamp and some are text.
When Maker gets data, and wants to Update record it works fine but only if all is 1:1. It fails if some of the records in HTTP request are Blank, or have “-” or for example “NA”.
Problem i assume is because airtable field is set for Currency or Number or Date and Make is sending update as Value=char/sign.

The other field was expecting a Date but Make was sending blank value and that failed as well, which is weird why it would. I suppose it should just skip value.

Is there a way, to “skip” sending Update on those fields that are not meeting some criteria, or change special chars to Blank/Empty, and somehow fix that side so it gets skipped?

Can anyone assist with any of this, please?

Thanks in advance

2

@Ash :raised_hands:

Hi there! I’m an AI that is still learning. I’m analyzing your message for typical information that might be missing for our community members to help. Can you provide more information about the limitations of the free plan on Make and Airtable? Also, can you provide more information about the fields in Airtable that are causing the issue?

Please take a camera-viewfinder-duotone screenshot of your scenario along with the relevant module configurations and share-all-duotoneshare the images here so that the community can help.

All of this helps us to get a deeper understanding of the challenge you face. :make:

3

  1. In the Make the Free plan also have access to all advanced functions. Not sure about Airtable.

  2. Use the Repeater module to process 5 records in each run, and the next run will take it from the 6th record. So use offset by Record ID. Along with this, you can save the last processed record id in Datastore so it will be easy if the scenario gets timeout (after 40 minutes).

Or

Use a filter to differentiate the records that are processed and got some column value updated.

  1. You can use various functions to correct the data format i.e. toNumber, toString or formatDate etc.

  2. Along with that ​I would recommend you use an error handler (prefer Break) to automatically handle errors, reattempt run and store the incomplete execution for your manual fix. Just right-click on the module and select add an error handler (then select break).

Learn more about error handlers here: https://www.make.com/en/help/errors

Learn about the break: https://www.make.com/en/help/errors/retry#using-the-break-directive

Watch this video to see how to implement Break in your scenario: Create Project Folders | Make - 4 November 2022 | Loom

1 Like