Create a JSON out of a pdf or jpg with chat gpt

Hello
I’m trying to create a scenario in which everytime a save a ticket or invoice in pdf format or jpg, Chat GPT can read it, extract key information and then create a record in Airtable with the expense.
First of all, I have developed the prompt and is working perfectly in Chat GPT. This is the prompt: Read the file {{8.id}} which is an invoice in pdf or jpg format and generate a JSON with the following information:

  • Issue date
  • Issuer’s VAT number
  • Issuing company name
  • Ticket or invoice number
  • Concept
  • Amount excluding VA
  • VAT rate (%
  • VAT
  • Total amount
    The date must be in DD.MM.YYYYY format.
    The amounts only with the decimals you do not have to separate the thousands, except the VAT rate (%) that goes without decimals. Decimals are indicated with a comma.

My make scenario is as follow:

  1. Watch files in Drive folder
  2. Download the file
  3. Upload the file with ChatGPT (Purpose: assistant)
  4. Read the file and extract the information in JSON. Role: System.
    The 4th step is failing. At the beginning it said it could not perform that kind of task, but now the output is zero.
    The next step will be to create the record in Airtable, but I’m still struggling with the 4th step.

Anyone can give me a clue?
I can share my scenario if needed.
Thanks and best regards

May peace be upon you @Natalia_Grech

Can you provide a screenshot of the module settings of the 4th step?

2 Likes

Sure!
Thanks for your help


1 Like

@Natalia_Grech I think you should send the message as an User not System.

2 Likes

I tried all of them (System, user, assistant) and nothing seemed to work.

Hi @Natalia_Grech

Please try giving the instructions of output as “User”, pass the data as “System” and give the prompt as “Assistant” to present the output as “json”.

If you require additional assistance, please don’t hesitate to reach out to us.
MSquare Support
Visit us here
Youtube Channel

1 Like

So, if I understood correctly I need to lay out 3 chatgpt modules, one for each step?
Or in one module with 3 items?
THanks

@Natalia_Grech

See the image for reference in one module.

If you require additional assistance, please don’t hesitate to reach out to us.
MSquare Support
Visit us here
Youtube Channel

1 Like

Ok, so it seems to quite worked, but I received this message as output:
“Sorry for any misunderstanding, but as a language AI model, I do not have the ability to access external files or read PDF or JPG documents. However, I could show you an example that will represent what your JSON output will look like.”
So what I don’t understand that the prompt works when I use it in chatgpt but not in Make.
Thanks again

@Natalia_Grech intenta creando un asistente en el API de OpenAI y dale las instrucciones al asistente para que haga lo que le pides.
mira este ejemplo:



Yep same here,

I don’t understand how to upload the file
The documentation says we need to use HTTP get a file
but it doesn’t help

https://www.make.com/en/help/app/openai-dall-e-chatgpt

All the files I try to upload seem to be corrupted and not usable by chatgpt

Hey @Rafael_Sanchez

I just wanted to let you know that I translated your question to English since that’s the official language of the community. By keeping the content in one language, we give everybody the chance to step in and help you.
Translation:

Try creating an assistant in the OpenAI API and give instructions to the assistant to do what you ask. Look at this example:

Note: Please read our Code of conduct and get familiar with the rules and guidelines that apply here in our community. This way we keep the community organized and neat for everybody.

Thank you very much

1 Like

Hi @Rafael_Sanchez
Where did you find the assistant? I don’t get any as options.
Do you have to create it first as a GPT?
Thanks for your help.
Best regards

Hi @Emile
I’m having issues with the upload. I use the option UPLOAD FILE out of the ChatGPT module.
Have you tried it?
Best regards

Hi @Rafael_Sanchez
I figured out how to create the assistant with the API, but I still get the same answer: It cannot process a pdf.
Thanks anyway.
Best regards

Yep the assistants are like GPTs but for the backend (as of my understanding). you need to create them here
https://platform.openai.com/assistants

As for the files, you can check the uploaded files here
https://platform.openai.com/files/
but the problem is that when we upload them with make it seem it’s not uploading properly the file

Someone here has done something with base64 for images, but it doesn’t seem to work on my end

@Rafael_Sanchez can you show us how you upload the file?

1 Like

Thanks @Emile
My issue is not with the upload, that seems to work fine. The problem is that ChatGPT says it cannot read pdf files.
BR

If you upload the file by hand and not via Make, you will see that you can read the pdf file.
It’s just that when you upload it with make it gets corrupted. (at least on my side, but maybe I’m using the Upload File module incorrectly)

Can you show me a screenshot of your OpenAi Upload File module ? (in step 3. Upload the file with ChatGPT (Purpose: assistant))

1 Like

@Natalia_Grech I saw the messages and you can do a couple of things.
if you want to read the PDF and are open to semi semi-automation scenario you could create a GPT that reads your PDF and then sends the information to a webhook and processes the information in integromat.

Another method is to create an assistant and you could use an “Upload file” module from ChatGPT, then use the file ID, and pass that ID to the assistant just like I show in my last picture.

please DM me for quick assistance on how to do it.

3 Likes

Hello @vendy ok, :slight_smile: I saw that @Natalia_Grech is giving instructions in Spanish and i assumed she may be a Spanish speaker, but you’re correct, I’ll keep communication in English

2 Likes