OpenAI analyzing Images or PDF and extracting data

I keep running into the problem that sometimes this workflow and Openai can accurately capture the pricing structure of a page but othertimes it makes it up. I try to make it as explicit as possible in the prompt but still to no avail. Is there something with my prompt or any other things i can try for OpenAI to accurately capture the pricing data? I tried downloading as PDF + OpenAI chat completion, I tried HTTPS, HTML to Text with text parser and then OpenAI analyze. What is the most accurate at capturing with data from an image or pdf orso that works 99% of the time/.

You are an AI assistant tasked with analyzing JPG files that contain website captures of AI tools. Your goal is to extract and summarize pricing information about each AI tool. When reviewing the provided JPG, follow these steps:

Pricing:

Summarize the pricing information for the subscription tiers, including high-level inclusions for each tier.
Present the information directly without phrases like “The image shows” or similar explanatory text.
Mention with each price the name of its tier.

Free Trial:

Detail any free trial offers, including the length of the trial and what it includes.
If there’s a free tier instead of or in addition to a free trial, describe its features.
Present the information directly without additional explanations about where the information is found in the image.
Specifically mention the billing cycle.

Discounts:

Note any discounts available, such as for annual payments or other special offers. Look for a toggle switch, button selection, or segmented control interface that specifies the billing cycle, as discounts are often associated with a certain billing cycle.
If applicable, mention the percentage or amount saved with discounts.
State this information directly without phrases like “This can be seen in…” or similar explanatory text.
Specifically mention the billing cycle.

Remember to focus solely on the information provided in the JPG and present it in a concise, direct manner. Don’t make up your own data, If any information is not available in the JPG, simply omit that part from your analysis rather than speculating or using external knowledge. If you encounter any ambiguities or need clarification on any point, please note this briefly in your response. Give the exact pricing per month orso with each subscription tier as is presented in the JPG. Don’t deviate from this

Example:

{
“pricing”: {
“Free Trial”: {
“price”: " Free Trial $0/M",
“features”: [“3,000 credits (one-time)”, “3 seats”, “5,000 knowledge pages”, “AI Assistants”, “Workflows”, “Core Integrations”]
},
“Starter”: {
“price”: " Starter $127/M",
“features”: [“15,000 credits per month”, “5 seats”, “50,000 knowledge pages”, “Unlimited Assistants”, “Unlimited Workflows”, “All Integrations (Slack, Notion, Sharepoint, …)”]
},
“Teams”: {
“price”: " Teams $424/M",
“features”: [“60,000 credits per month”, “10 seats”, “150,000 knowledge pages”, “Unlimited Assistants”, “Unlimited Workflows”, “All Knowledge Base Integrations (Slack, Notion, Sharepoint, …)”, “Dedicated Slack support”, “1h AI Training Session”]
},
“Enterprise”: {
“price”: “Enterprise Custom”,
“features”: [“Custom credits per month”, “Custom seats”, “Custom knowledge pages”, “Unlimited Assistants”, “Unlimited Workflows”, “All Knowledge Base Integrations (Slack, Notion, Sharepoint, …)”, “White glove setup and onboarding”, “Advanced permission controls”, “Single Sign-On (SSO) capability”, “Dedicated AI Implementation Expert”, “Personalized AI training sessions”, “Custom integrations”, “Live syncing with connections”, “Workflow deployments”, “Document Verification AI”, “ROI Report”]
}
},
“freeTrial”: {
“offers”: “Free tier”,
“features”: [“3,000 credits (one-time)”, “3 seats”, “5,000 knowledge pages”, “AI Assistants”, “Workflows”, “Core Integrations”]
},
“discounts”: “15% discount for yearly billing”
}
}



This example is only slightly off but othertimes its way off and others its perfect

Did you select “JSON Object” format of a “Create a Completion” module, under the Advanced Settings?

Screenshot_2024-07-29_130710

samliewrequest private consultation

Join the Make Fans Discord server to chat with other makers!

Where is the this setting, within which module? Cant seem to find it Google Drive or OpenAI Image Vision

Hi @Sam_Van_Leeuwen ,

Welcome to Make Community!

The image analyzing module is not very good at creating a JSON structure. So, you need to use another GPT: Create Completion Module. You can pass the output from the Image Analyzing module to it. You can then provide a prompt to generate a proper JSON structure from the extracted details.

In this module, you can even enable the JSON format.

If you require additional assistance, please don’t hesitate to reach out to us.
MSquare Support
| Book Live Implementation
Visit us here
Youtube Channel

but is this than also better at taking the data from a webpage on pricing? Like this way analyzing a webpage turned pdf is better than webpage turned jpg and using openai image vision?

Sam, I’ve made an example scenario for you. Feel free to amend as necessary.

Module Export

You can copy and paste this module export into your scenario. This will paste the modules shown in my screenshots above.

  1. Copy the JSON code below by clicking the copy button when you mouseover the top-right of the code block
    Screenshot_2024-01-17_200117

  2. Enter your scenario editor. Press ESC to close any dialogs. Press CTRLV (paste keyboard shortcut for Windows) to paste directly in the canvas.

  3. Click on each imported module and save it for validation. You may be prompted to remap some variables and connections.

Click to Expand Module Export Code

JSON - Copy and Paste this directly in the scenario editor

{
 "subflows": [
  {
   "flow": [
    {
     "id": 109,
     "module": "http:ActionSendData",
     "version": 3,
     "parameters": {
      "handleErrors": true,
      "useNewZLibDeCompress": true
     },
     "mapper": {
      "url": "https://www.jasper.ai/pricing",
      "serializeUrl": false,
      "method": "get",
      "headers": [],
      "qs": [],
      "bodyType": "raw",
      "parseResponse": false,
      "authUser": "",
      "authPass": "",
      "timeout": "",
      "shareCookies": false,
      "ca": "",
      "rejectUnauthorized": true,
      "followRedirect": true,
      "useQuerystring": false,
      "gzip": true,
      "useMtls": false,
      "contentType": "text/html",
      "data": "",
      "followAllRedirects": false
     },
     "metadata": {
      "designer": {
       "x": 1713,
       "y": -1275
      },
      "restore": {
       "expect": {
        "method": {
         "mode": "chose",
         "label": "GET"
        },
        "headers": {
         "mode": "chose",
         "collapsed": true
        },
        "qs": {
         "mode": "chose",
         "collapsed": true
        },
        "bodyType": {
         "label": "Raw"
        },
        "parseResponse": {
         "collapsed": true
        },
        "contentType": {
         "label": "HTML (text/html)"
        },
        "data": {
         "collapsed": true
        }
       }
      },
      "parameters": [
       {
        "name": "handleErrors",
        "type": "boolean",
        "label": "Evaluate all states as errors (except for 2xx and 3xx )",
        "required": true
       },
       {
        "name": "useNewZLibDeCompress",
        "type": "hidden"
       }
      ],
      "expect": [
       {
        "name": "url",
        "type": "url",
        "label": "URL",
        "required": true
       },
       {
        "name": "serializeUrl",
        "type": "boolean",
        "label": "Serialize URL",
        "required": true
       },
       {
        "name": "method",
        "type": "select",
        "label": "Method",
        "required": true,
        "validate": {
         "enum": [
          "get",
          "head",
          "post",
          "put",
          "patch",
          "delete",
          "options"
         ]
        }
       },
       {
        "name": "headers",
        "type": "array",
        "label": "Headers",
        "spec": [
         {
          "name": "name",
          "label": "Name",
          "type": "text",
          "required": true
         },
         {
          "name": "value",
          "label": "Value",
          "type": "text"
         }
        ]
       },
       {
        "name": "qs",
        "type": "array",
        "label": "Query String",
        "spec": [
         {
          "name": "name",
          "label": "Name",
          "type": "text",
          "required": true
         },
         {
          "name": "value",
          "label": "Value",
          "type": "text"
         }
        ]
       },
       {
        "name": "bodyType",
        "type": "select",
        "label": "Body type",
        "validate": {
         "enum": [
          "raw",
          "x_www_form_urlencoded",
          "multipart_form_data"
         ]
        }
       },
       {
        "name": "parseResponse",
        "type": "boolean",
        "label": "Parse response",
        "required": true
       },
       {
        "name": "authUser",
        "type": "text",
        "label": "User name"
       },
       {
        "name": "authPass",
        "type": "password",
        "label": "Password"
       },
       {
        "name": "timeout",
        "type": "uinteger",
        "label": "Timeout",
        "validate": {
         "max": 300,
         "min": 1
        }
       },
       {
        "name": "shareCookies",
        "type": "boolean",
        "label": "Share cookies with other HTTP modules",
        "required": true
       },
       {
        "name": "ca",
        "type": "cert",
        "label": "Self-signed certificate"
       },
       {
        "name": "rejectUnauthorized",
        "type": "boolean",
        "label": "Reject connections that are using unverified (self-signed) certificates",
        "required": true
       },
       {
        "name": "followRedirect",
        "type": "boolean",
        "label": "Follow redirect",
        "required": true
       },
       {
        "name": "useQuerystring",
        "type": "boolean",
        "label": "Disable serialization of multiple same query string keys as arrays",
        "required": true
       },
       {
        "name": "gzip",
        "type": "boolean",
        "label": "Request compressed content",
        "required": true
       },
       {
        "name": "useMtls",
        "type": "boolean",
        "label": "Use Mutual TLS",
        "required": true
       },
       {
        "name": "contentType",
        "type": "select",
        "label": "Content type",
        "validate": {
         "enum": [
          "text/plain",
          "application/json",
          "application/xml",
          "text/xml",
          "text/html",
          "custom"
         ]
        }
       },
       {
        "name": "data",
        "type": "buffer",
        "label": "Request content"
       },
       {
        "name": "followAllRedirects",
        "type": "boolean",
        "label": "Follow all redirect",
        "required": true
       }
      ]
     }
    },
    {
     "id": 110,
     "module": "openai-gpt-3:CreateCompletion",
     "version": 1,
     "parameters": {
      "__IMTCONN__": 107818
     },
     "mapper": {
      "select": "chat",
      "max_tokens": "0",
      "temperature": "0.15",
      "top_p": "1",
      "n_completions": "1",
      "response_format": "json_object",
      "model": "gpt-4o-mini",
      "messages": [
       {
        "role": "assistant",
        "content": "Pricing:\nSummarize the pricing information for the subscription tiers, including high-level inclusions for each tier.\nPresent the information directly without phrases like \"The image shows\" or similar explanatory text.\nMention with each price the name of its tier.\n\nFree Trial:\nDetail any free trial offers, including the length of the trial and what it includes.\nIf there’s a free tier instead of or in addition to a free trial, describe its features.\nPresent the information directly without additional explanations about where the information is found in the image.\nSpecifically mention the billing cycle.\n\nDiscounts:\nNote any discounts available, such as for annual payments or other special offers. Look for a toggle switch, button selection, or segmented control interface that specifies the billing cycle, as discounts are often associated with a certain billing cycle.\nIf applicable, mention the percentage or amount saved with discounts.\nState this information directly without phrases like \"This can be seen in…\" or similar explanatory text.\nSpecifically mention the billing cycle.\n\nRemember to focus solely on the information provided and present it in a concise, direct manner. Don’t make up your own data, If any information is not available, simply omit that part from your analysis rather than speculating or using external knowledge. If you encounter any ambiguities or need clarification on any point, please note this briefly in your response. Give the exact pricing per month or so with each subscription tier as is presented. Don’t deviate from this.\n\nReturn the result as JSON, similar to the following example:\n\n{\n\"pricing\": {\n\"Free Trial\": {\n\"price\": \" Free Trial $0/M\",\n\"features\": [\"3,000 credits (one-time)\", \"3 seats\", \"5,000 knowledge pages\", \"AI Assistants\", \"Workflows\", \"Core Integrations\"]\n},\n\"Starter\": {\n\"price\": \" Starter $127/M\",\n\"features\": [\"15,000 credits per month\", \"5 seats\", \"50,000 knowledge pages\", \"Unlimited Assistants\", \"Unlimited Workflows\", \"All Integrations (Slack, Notion, Sharepoint, …)\"]\n},\n\"Teams\": {\n\"price\": \" Teams $424/M\",\n\"features\": [\"60,000 credits per month\", \"10 seats\", \"150,000 knowledge pages\", \"Unlimited Assistants\", \"Unlimited Workflows\", \"All Knowledge Base Integrations (Slack, Notion, Sharepoint, …)\", \"Dedicated Slack support\", \"1h AI Training Session\"]\n},\n\"Enterprise\": {\n\"price\": \"Enterprise Custom\",\n\"features\": [\"Custom credits per month\", \"Custom seats\", \"Custom knowledge pages\", \"Unlimited Assistants\", \"Unlimited Workflows\", \"All Knowledge Base Integrations (Slack, Notion, Sharepoint, …)\", \"White glove setup and onboarding\", \"Advanced permission controls\", \"Single Sign-On (SSO) capability\", \"Dedicated AI Implementation Expert\", \"Personalized AI training sessions\", \"Custom integrations\", \"Live syncing with connections\", \"Workflow deployments\", \"Document Verification AI\", \"ROI Report\"]\n}\n},\n\"freeTrial\": {\n\"offers\": \"Free tier\",\n\"features\": [\"3,000 credits (one-time)\", \"3 seats\", \"5,000 knowledge pages\", \"AI Assistants\", \"Workflows\", \"Core Integrations\"]\n},\n\"discounts\": \"15% discount for yearly billing\"\n}\n}"
       },
       {
        "role": "user",
        "content": "{{stripHTML(toString(109.data))}}"
       }
      ]
     },
     "metadata": {
      "designer": {
       "x": 1957,
       "y": -1275,
       "name": "OpenAI Completion"
      },
      "restore": {
       "parameters": {
        "__IMTCONN__": {
         "collapsed": true,
         "label": "OpenAI",
         "data": {
          "scoped": "true",
          "connection": "openai-gpt-3"
         }
        }
       },
       "expect": {
        "select": {
         "label": "Create a Chat Completion (GPT Models)"
        },
        "max_tokens": {
         "collapsed": true
        },
        "temperature": {
         "collapsed": true
        },
        "top_p": {
         "collapsed": true
        },
        "n_completions": {
         "collapsed": true
        },
        "frequency_penalty": {
         "collapsed": true
        },
        "presence_penalty": {
         "collapsed": true
        },
        "logit_bias": {
         "mode": "chose",
         "collapsed": true
        },
        "response_format": {
         "mode": "chose",
         "label": "JSON Object"
        },
        "seed": {
         "collapsed": true
        },
        "stop": {
         "mode": "chose",
         "collapsed": true
        },
        "additionalParameters": {
         "mode": "chose",
         "collapsed": true
        },
        "model": {
         "mode": "chose",
         "label": "gpt-4o-mini (system)"
        },
        "messages": {
         "mode": "chose",
         "items": [
          {
           "role": {
            "mode": "chose",
            "label": "Assistant"
           }
          },
          {
           "role": {
            "mode": "chose",
            "label": "User"
           }
          }
         ]
        }
       }
      },
      "parameters": [
       {
        "name": "__IMTCONN__",
        "type": "account:openai-gpt-3",
        "label": "Connection",
        "required": true
       }
      ],
      "expect": [
       {
        "name": "select",
        "type": "select",
        "label": "Select Method",
        "required": true,
        "validate": {
         "enum": [
          "chat",
          "prompt"
         ]
        }
       },
       {
        "name": "max_tokens",
        "type": "number",
        "label": "Max Tokens",
        "required": true
       },
       {
        "name": "temperature",
        "type": "number",
        "label": "Temperature",
        "validate": {
         "max": 2,
         "min": 0
        }
       },
       {
        "name": "top_p",
        "type": "number",
        "label": "Top P",
        "validate": {
         "max": 1,
         "min": 0
        }
       },
       {
        "name": "n_completions",
        "type": "number",
        "label": "Number"
       },
       {
        "name": "frequency_penalty",
        "type": "number",
        "label": "Frequency Penalty",
        "validate": {
         "max": 2,
         "min": -2
        }
       },
       {
        "name": "presence_penalty",
        "type": "number",
        "label": "Presence Penalty",
        "validate": {
         "max": 2,
         "min": -2
        }
       },
       {
        "name": "logit_bias",
        "type": "array",
        "label": "Token Probability",
        "spec": {
         "spec": [
          {
           "name": "token",
           "type": "text",
           "label": "Token ID",
           "required": true
          },
          {
           "name": "probability",
           "type": "number",
           "label": "Probability",
           "required": true,
           "validate": {
            "max": 100,
            "min": -100
           }
          }
         ],
         "type": "collection",
         "label": "Token Probability",
         "name": "value"
        }
       },
       {
        "name": "response_format",
        "type": "select",
        "label": "Response Format",
        "validate": {
         "enum": [
          "text",
          "json_object"
         ]
        }
       },
       {
        "name": "seed",
        "type": "integer",
        "label": "Seed"
       },
       {
        "name": "stop",
        "type": "array",
        "label": "Stop Sequences",
        "validate": {
         "maxItems": 4
        },
        "spec": {
         "type": "text",
         "label": "Stop Sequence",
         "name": "value"
        }
       },
       {
        "name": "additionalParameters",
        "type": "array",
        "label": "Other Input Parameters",
        "spec": {
         "spec": [
          {
           "name": "key",
           "type": "text",
           "label": "Parameter Name",
           "required": true
          },
          {
           "name": "type",
           "type": "select",
           "label": "Input Type",
           "options": [
            {
             "label": "Text",
             "value": "text",
             "nested": [
              {
               "name": "value",
               "type": "text",
               "label": "Parameter Value"
              }
             ],
             "default": true
            },
            {
             "label": "Number",
             "value": "number",
             "nested": [
              {
               "name": "value",
               "type": "number",
               "label": "Parameter Value"
              }
             ]
            },
            {
             "label": "Boolean",
             "value": "boolean",
             "nested": [
              {
               "name": "value",
               "type": "boolean",
               "label": "Parameter Value"
              }
             ]
            },
            {
             "label": "Date",
             "value": "date",
             "nested": [
              {
               "name": "value",
               "type": "date",
               "label": "Parameter Value"
              }
             ]
            },
            {
             "label": "Any",
             "value": "any",
             "nested": [
              {
               "name": "value",
               "type": "any",
               "label": "Parameter Value"
              }
             ]
            }
           ]
          }
         ],
         "type": "collection",
         "label": "Input Parameter",
         "name": "value"
        }
       },
       {
        "name": "model",
        "type": "select",
        "label": "Model",
        "required": true
       },
       {
        "name": "messages",
        "type": "array",
        "label": "Messages",
        "required": true,
        "spec": {
         "spec": [
          {
           "name": "role",
           "type": "select",
           "label": "Role",
           "options": [
            {
             "label": "System",
             "value": "system"
            },
            {
             "label": "User",
             "value": "user"
            },
            {
             "label": "Assistant",
             "value": "assistant"
            }
           ],
           "required": true
          },
          {
           "name": "content",
           "type": "text",
           "label": "Message Content"
          }
         ],
         "type": "collection",
         "label": "Message",
         "name": "value"
        }
       }
      ],
      "advanced": true
     }
    },
    {
     "id": 111,
     "module": "json:ParseJSON",
     "version": 1,
     "parameters": {
      "type": ""
     },
     "mapper": {
      "json": "{{110.result}}"
     },
     "metadata": {
      "designer": {
       "x": 2201,
       "y": -1274,
       "messages": [
        {
         "category": "last",
         "severity": "warning",
         "message": "A transformer should not be the last module in the route."
        }
       ]
      },
      "restore": {
       "parameters": {
        "type": {
         "label": "Choose a data structure"
        }
       }
      },
      "parameters": [
       {
        "name": "type",
        "type": "udt",
        "label": "Data structure"
       }
      ],
      "expect": [
       {
        "name": "json",
        "type": "text",
        "label": "JSON string",
        "required": true
       }
      ]
     }
    }
   ]
  }
 ],
 "metadata": {
  "version": 1
 }
}

Hope this helps! Let me know if there are any further issues!

samliewrequest private consultation

Join the Make Fans Discord server to chat with other makers!

Omg thank you, it does not work on every websote perfectly still but atleast no extra 3rd party module is needed that will cost money foe the same repsones. One last thing is how do I get it to mention the name of the pricing or subscription tier aswell so taht when o put this data back into Airtable not only the price and featuers are pushed over but also the corresponding name of the tier?

You can simply amend the prompt and example JSON so that OpenAI knows it should try to match that information on the website.

Good luck!

samliewrequest private consultation

Join the unofficial Make Discord server to chat with us!