Split a large text file

Hi,

I have a text file that contain arround 40 000 lines.
I need to upload some of them into a airtable database but it take approximatively 45min to upload 2 300 lines in my airtable database.
So i need to split my work to do it

  1. Download the file from https

  2. Split my file
    Each file must be arround 2200 lines

  3. Create as many job as needed to load the files

Do you think that’s may be realistic ?
Do you think that an other way might be better ?
Thanks

Welcome to the Make community!

Do you need to do this as a one-off uploading, or need to do it monthly?

If once-off, then AIrtable may already have import tools that may be more suitable for this task.

samliewrequest private consultation

Join the Make Fans Discord server to chat with other makers!

Hi @samliew ,

I need a monthly uploading. each month a new file is uploaded most of the lines are the same but we don’t know witch one will be updated or deleted

thanks

I have realised something short here.
The array is split up and then sent to a second scenario in the desired portions.
I hope it helps.

send_to_db.json (1.7 KB)
split_file.json (15.9 KB)