I’m trying to build an automation for an accounting firm where they receive invoices and bills from clients that need to be automatically recorded as sales in the books.
My test scenario receives images of invoices/bills by whatsapp which is then passed onto Google OCR. The text is then passed onto the openAI Transform Text to Structured Data module. After that I pretty much need to record the customers, items, etc onto Zoho Books. Most of this is working fine, except the OpenAI module which consistently fails to extract data for all items.
This is really weird as I have included an instruction in the prompt to count all items in the invoice, which it does perfectly. It just doesn’t give me the right data. Below is the prompt I’m using, which is mostly generated with ChatGPT itself -
Extract all line items from the invoice along with the additional following details:
- Invoice Number
- Date
- Vendor Name
- Line Items (Description, Quantity, Unit Price, Total) (Each item on the invoice)
- Customer Name
- Customer Address
- Item Count
For example -
Text:
Invoice Number: 12345,
Date: 2024-06-01,
Vendor: ABC Supplies,
Item: Widget A, Quantity: 10, Unit Price: 5, Total: 50;
Item: Widget B, Quantity: 12, Unit Price: 2, Total: 24
Output JSON:
{
“invoice_number”: “12345”,
“date”: “2024-06-01”,
“vendor_name”: “ABC Supplies”,
“line_items”: [
{
“description”: “Widget A”,
“quantity”: 10,
“unit_price”: 5,
“total”: 50
}
{
“description”: “Widget B”,
“quantity”: 12,
“unit_price”: 2,
“total”: 24
}
]
}
Make sure details for every item are processed. If item count is 6, ensure data is extracted for all 6 items.
You can see I have actually tried to force it to output the right data, to no avail. The scenario works correctly around 50% of the time.
Things I have already tried-
-Tests with different models. Currently on 4o.
-Tried around 6-8 different modifications, removing and adding examples, etc
I could try swapping Google OCR with OpenAI but that would be way too expensive for an accounting firm with hundreds of clients that send hundreds of invoices/bills daily.
I have included a video of the structure but I’m quite happy with it and it works perfectly when it…well, works perfectly.
