Best practice for multi-cycle request building and then referencing to it

[fruits]

Issue

What is the best practice to address former cycles when building up a query?

Solution and Context

I am using an api to enhance entries in my airtable data. The api i am using offers a bulk method i need to address, this method allows me to ask for the results for ten entries at once.

To solve this i built the following flow:

1. Retrieve all entries from the specific airtable table
2. On the first value start building the json string, on value 2-9 add up the values and if you reached the 10th value add the ending elements of the json string and do the request.
3. Request the results for the first 10 values, then the second 10 values and so on.

The result is that make.com runs through the flow for 10 cycles, then continues for the bottom route to sent the request.

My Issue: I now need to update my airtable with my results. I can solve this by using the id i got in my response - using a “Find Record” module, followed by a “Update Record” module. However it feels incredibly a waste of resources if i would not benefit from the fact, that i know the 10 values i want to update based on the fact that this were the 9 former and the current cycle of the flow path i am in. What would be the best practice to address these values since with the variable references of make.com it seems i can only access the current one. Store them all in an array?

Screenshots

The only module before this flow is a “Search Entries” Airtable Module

[samliew]

There are too many get and set variables here.

Without knowing they they are really doing it’s hard to suggest possible optimisations.

Could you provide example bundles coming from the Trigger module, as well as screenshots of each module and filter?

1. Screenshots of module fields and filters

Please share screenshots of relevant module fields and filters in question? It would really help other community members to see what you’re looking at.

You can upload images here using the Upload icon in the text editor:

2. Scenario blueprint

Please export the scenario blueprint file to allow others to view the mappings and settings. At the bottom of the scenario editor, you can click on the three dots to find the Export Blueprint menu item.

^{(Note: Exporting your scenario will not include private information or keys to your connections)}

Uploading it here will look like this:

blueprint.json (12.3 KB)

3. And most importantly, Input/Output bundles

Please provide the input and output bundles of the trigger/iterator/aggregator modules by running the scenario (or get from the scenario History tab), then click the white speech bubble on the top-right of each module and select “Download input/output bundles”.

A.

Save each bundle contents in your text editor as a bundle.txt file, and upload it here into this discussion thread.

Uploading them here will look like this:

module-1-output-bundle.txt (12.3 KB)

B.

If you are unable to upload files on this forum, alternatively you can paste the formatted bundles in this manner:

• Either add three backticks ``` before and after the code, like this:

  ```
  <sup>input/output bundle content goes here</sup>
  ```

• Or use the format code button in the editor:
  

Providing the input/output bundles will allow others to replicate what is going on in the scenario even if they do not use the external service.

Following these steps will allow others to assist you here. Thanks!

samliew – request private consultation

Join the Make Fans Discord server to chat with other makers!

[fruits]

To be honest, i have a hard time exporting Input/Output Bundles in full since i am handling personal data, i tried my best to now provide screenshots and suitable examples.

At first again the complete workflow:

Now single screenshots of the modules, with input and output bundles:
Module 50 - Search for Records

Input

[
    {
        "base": "appcqTJdIoZOmsGO0",
        "view": "viw6iM8V2hbf7gtpZ",
        "table": "tblNkL7vZj9eX5HtB",
        "fields": [
            "contact_id",
            "Contact",
            "Company Lead",
            "saturn_contact_id",
        ],
        "maxRecords": 10,
        "useColumnId": false
    }
]

Output

[
    {
        "contact_id": 81380,
        "Company Lead": [
            "receXucwD8q42N34l"
        ],
        "Contact": "email_not_unlocked@domain.com",
        "saturn_contact_id": "6641d38a89a9890001939221",
        "id": "recMNPkCO0EID0GNq",
        "createdTime": "2024-05-23T23:00:33.000Z",
        "__IMTLENGTH__": 10,
        "__IMTINDEX__": 1
    },
    {
        "contact_id": 82737,
        "Company Lead": [
            "rechZoQUc81kpIxK1"
        ],
        "Contact": "email_not_unlocked@domain.com",
        "saturn_contact_id": "5570ab6073696422b32e0b00",
        "id": "recu8Cdi2JHJ6bKJd",
        "createdTime": "2024-05-24T10:05:22.000Z",
        "__IMTLENGTH__": 10,
        "__IMTINDEX__": 10
    }
]

Module 52 - Increment Function

Module 53 - Set Variable
(Is called in the first run of each set)

Input

[
    {
        "name": "query_string",
        "scope": "roundtrip",
        "value": "[{\"id\" : \"6641d38a89a9890001939221\"}"
    }
]

Output

[
    {
        "query_string": "[{\"id\" : \"6641d38a89a9890001939221\"}"
    }
]

Module 57 - Get the Variable
(Is called in the 2-10th run of each set)

Input

[
    {
        "name": "query_string"
    }
]

Output

[
    {
        "query_string": "[{\"id\" : \"6641d38a89a9890001939221\"}"
    }
]

Module 58 - Set the Variable
(Is also called in the 2-10th run of each set)

Input

[
    {
        "name": "query_string",
        "scope": "roundtrip",
        "value": "[{\"id\" : \"6641d38a89a9890001939221\"},{\"id\" : \"60b74767f9b4a30001ab70a8\"},{\"id\" : \"60fc2edc170a7e000122219b\"},{\"id\" : \"60d32a8ed1d1f20001f7b13d\"}"
    }
]

Output

[
    {
        "query_string": "[{\"id\" : \"6641d38a89a9890001939221\"},{\"id\" : \"60b74767f9b4a30001ab70a8\"},{\"id\" : \"60fc2edc170a7e000122219b\"},{\"id\" : \"60d32a8ed1d1f20001f7b13d\"}"
    }
]

Module 60 - Set the Variable
(Only called after the 10th run to close the string)

Input

[
    {
        "name": "query_string",
        "scope": "roundtrip",
        "value": "[{\"id\" : \"6641d38a89a9890001939221\"},{\"id\" : \"60b74767f9b4a30001ab70a8\"},{\"id\" : \"60fc2edc170a7e000122219b\"},{\"id\" : \"60d32a8ed1d1f20001f7b13d\"},{\"id\" : \"57de713ea6da987ae8cd2601\"},{\"id\" : \"6411dc5eae252e0001569510\"},{\"id\" : \"5f4b62b8eab77a0001fa65b2\"},{\"id\" : \"5f9ee7e405bff60001e832d1\"},{\"id\" : \"61178fa3bbe2750001905e55\"},{\"id\" : \"5570ab6073696422b32e0b00\"}]"
    }
]

Output

[
    {
        "query_string": "[{\"id\" : \"6641d38a89a9890001939221\"},{\"id\" : \"60b74767f9b4a30001ab70a8\"},{\"id\" : \"60fc2edc170a7e000122219b\"},{\"id\" : \"60d32a8ed1d1f20001f7b13d\"},{\"id\" : \"57de713ea6da987ae8cd2601\"},{\"id\" : \"6411dc5eae252e0001569510\"},{\"id\" : \"5f4b62b8eab77a0001fa65b2\"},{\"id\" : \"5f9ee7e405bff60001e832d1\"},{\"id\" : \"61178fa3bbe2750001905e55\"},{\"id\" : \"5570ab6073696422b32e0b00\"}]"
    }
]

Module 19 - Call the API

Input

[
    {
        "ca": null,
        "qs": [],
        "url": "https://api.saturn.io/v6/assets/bulk",
        "data": "{\n    \"api_key\": \"-\",\n    \"reveal_personal_emails\": true,\n    \"details\" : [{\"id\" : \"6641d38a89a9890001939221\"},{\"id\" : \"60b74767f9b4a30001ab70a8\"},{\"id\" : \"60fc2edc170a7e000122219b\"},{\"id\" : \"60d32a8ed1d1f20001f7b13d\"},{\"id\" : \"57de713ea6da987ae8cd2601\"},{\"id\" : \"6411dc5eae252e0001569510\"},{\"id\" : \"5f4b62b8eab77a0001fa65b2\"},{\"id\" : \"5f9ee7e405bff60001e832d1\"},{\"id\" : \"61178fa3bbe2750001905e55\"},{\"id\" : \"5570ab6073696422b32e0b00\"}]\n}",
        "gzip": true,
        "method": "post",
        "headers": [],
        "timeout": null,
        "useMtls": false,
        "authPass": null,
        "authUser": null,
        "bodyType": "raw",
        "contentType": "application/json",
        "serializeUrl": false,
        "shareCookies": false,
        "parseResponse": false,
        "followRedirect": true,
        "useQuerystring": false,
        "followAllRedirects": false,
        "rejectUnauthorized": true
    }
]

Output

[
    {
        "status": "success",
        "error_code": null,
        "error_message": null,
        "total_requested_enrichments": 10,
        "unique_enriched_records": 9,
        "missing_records": 1,
        "credits_consumed": 9,
        "matches": [
            {
                "id": "6641d38a89a9890001939221",
                "email": example@example.com,
            },
            {
                "id": "60b74767f9b4a30001ab70a8",
                "email": example2@example.com,
            },
        ]
    }
]

[fruits]

To rephrase my original question, as you can see i am getting back values with the saturn_contact_id and the email information. Now i need to update airtable using an airtable-id. From module 50 i have all airtable-id’s and their respective saturn_contact_ids, so if i had a chance to store these over multiple cycles until after the 10th run i have results, my issue would be solved.

I need a way of storing which Airtable-Id belongs to which saturn_contact_id coming from the first request, to later be able to use this airtable-id to update the email address.