How to include both images and files in OpenAI Generate a response module

:bullseye: What is your goal?

Include both PDF files and images in one run of OpenAI Make a response module.

:thinking: What is the problem & what have you tried?

Currently, I can select File prompt or Image prompt in Prompt type when setting up the module. However, I need to use both PDFs and images in one request in the module.
When selecting File prompt and sending an jpg or png image in, the run ends with an error of wrong file type.

:clipboard: Error messages or input/output bundles

[400] input[0].content[0].file_data: Invalid file data: ‘input[0].content[0].file_data’. Expected a base64-encoded data URL with a valid file MIME type (e.g. ‘data:text/plain;base64,SGVsbG8sIFdvcmxkIQ==’), but got unsupported MIME type ‘image/jpeg’. Please see https://platform.openai.com/docs/assistants/tools/file-search#supported-files for supported file types.

This is probably a limit in the Make module.

Right now, the module lets you choose only one type:

  • File prompt for PDFs/docs

  • Image prompt for JPG/PNG

So if you want to send PDF + image together in one request, the easier fix is to use an HTTP module instead of the normal OpenAI module.

So the short answer is: the standard module does not handle both together well, but the HTTP call can.

1 Like

If you select Prompt Type as Advanced Input, you then have the option to include a list of input types (eg file and Image).

It’s not a simple interface unfortunately, as it mirrors the OpenAI API structure which itself is quite complex!

Thanks! I found this, but thought that this wouldn’t work for me as I need to input undetermined number of files, unfortunately…

You can still map the array there, as long as it follows the expected structure. Just toggle the map button next to Content and not down bellow next to the individual items.

1 Like