Google Vertex AI (Gemini) - Analyze Image/Video (gemini-pro-vision)

Hi Makers,

We are trying to create an automation process that scrapes all the relevant data from competitors’ Instagram accounts, including media URLs, like counts, comment counts, and Instagram permalinks. Additionally, we would like to obtain descriptions of the posts. We are able to achieve this for images using OpenAI’s Analyze Image feature, but it does not work for videos. To address this issue, we explored different options and found Google Vertex AI (Gemini), which offers Image/Video Analysis.

We have set up every possible step for the module and tried all the options in the regions, roles, file upload types, MIME types, and file data, but nothing seems to work. We frequently encounter “400” errors, indicating that the request contains an invalid argument.


Snímka obrazovky 2024-05-09 104211

I attempted to reach out to support, but it is not possible to select Google Vertex AI (Gemini) as the name of the app when submitting a support request.

Does anyone have experience setting up Google Vertex AI in general? I have a feeling that the Google Vertex App either does not work at all or requires very complex setup steps.

Curious if you got an answer to this? Looking for a similar solution

Hi,

I did not, but I found a solution :blush:. You need to add the HTTP module (Get a file) with the media URL from Instagram before Gemini. The setup in Gemini could look like this:

2 Likes

Hello there @Samuel1 :wave:

Great to hear that you’ve found a solution to your struggles!

We appreciate you sharing the solution with us and maintaining our community organized. :pray:

1 Like

Hi Samuel,

Thanks for the solution.
Is Gemini giving you good responses?

Because I tried analyzing some videos and Gemini always hallucinates; the answers are very far from the content of my videos.

Perhaps I missed something in the settings…