Split csv or process 10 bundles each time

[Javier_Vinas]

Hi automation folks!

I’m stuck with one scenario and I don’t know how to approach the problem.

Basically I want to extract a csv file from a Gmail attachtment, convert it to a specific JSON structure and upload it to an Airtable base. I managed to do all the steps and it’s working, but Airtable API request can only accept 10 records max in each API call.

So I guess I need to find a way to:

• Split the csv file into chunks of 10 rows OR only process 10 bundles of the JSON file each time
• Loop through that until the entire file is completed

Here’s the scenario for your reference:

JSON module with 25 records - I will only need to process 10, and then 10, and then 5. If that makes sense.

Appreciate some guidance on this

Cheers

[Francisco_Fontes]

Hello, @Javier_Vinas , I share a way to do what you ask.

Process executed

99 records has the csv

Iterate the array every 10 elements

Cut the array every 10 elements

Create json to enter Airtable

Bulk data entry on Airtable

[Javier_Vinas]

Hi Francisco,

Brilliant stuff! Thank you so much for laying out all the instructions.

I haven’t checked the solution on my scenario - but I’m pretty confident that this will work. I marked it as a solution.

Gracias maestro

[gneuman]

Hello, this is how I will solve the problem. It need something to code, but the benefits are that you can upload more records each run.

1. Create a Huge JSON file with the 100 records (or more) and pass to Airtable

2. In Airtable create a simple automation (New Record) to Run a JavaScript, for this you will need to be able to run I think in the pro you can do it.

3. This is the Script you need to pass.

let params = input.config();
let contentId = params.contentId;
let jsonCSV = params.jsonCSV;

let records = JSON.parse(jsonCSV);

let createArr = records.map(obj => {
return {
fields: {
“IP”: obj.IP,
“Sexo”: obj.Sexo,
“Email”: obj.Email,
“Nombre”: obj.Nombre,
“Apellido”: obj.Apellido,
}
}
})

let table = base.getTable(“Records”);

while(createArr.length > 0){
await table.createRecordsAsync(createArr.slice(0,50));
createArr = createArr.slice(50);
}

I had recorded a video (in Spanish) of the process you can see at ¿Cómo crear más de 50 récords al mismo tiempo en Airtable? - YouTube