Scraping Bee Integration Runtime Error 400

Hello there,

I want to create an integration in which I automatically check the RSS field of the OpenAI Blog, than scrape the content via the URL with the tool ScrapingBee.

Unfortunately I get the following error:
" RuntimeError


Does anyone encountered a similar problem or knows how to fix it?

Thank you in advance!

With best regards,

I believe OpenAI has implemented anti-scrape measures on their website and blog.

I would use ScrapeNinja “Scrape (Real browser)” to emulate a real person visiting the site using a web browser, as client-side JavaScript needs to run to retrieve the data.

You CANNOT use normal scraping apps like ScrapingBee, or HTTP “Make a request” module to fetch this data.



I didn’t know that :0
Thank you for your answer! I will try out ScapeNinja with hopefully positive results!!

Have a great day!

1 Like