Hello, how to scrape a secured anti-bot website?
I’ve tried to scrape product pages from this website with HTTP, ScrapeNinja, Airtop but impossible :
https://www.cedeo.fr/search?q=4758685
Any idea?
I’m getting “Please enable JS and disable any ad blocker” message error with HTTP module and ScrapeNinja :
https://community.make.com/t/login-and-parse-html-webpage-please-enable-js-and-disable-any-ad-blocker
And this unlock screen with Airtop :
Hey @Garry_V,
I would suggest building an API connection using the HTTP module on the Bright data service.
It is a really great tool to perform scraping and it has rebuilt solutions to scrape easily big platforms like social media, Amazon and etc.
Thanks,
Dimitris Goudis
Thank you. I know, but this provider Cédéo I already work with has no API.
@Garry_V
Zyte API handles 90% of captcha websites, if not, ScrapingBee almost definitely does (but costs more)
Of course the Web Unlocker API of Bright Data would probably work as well
Hey @Garry_V,
This is why you need scraping service and brigth data that has api so you can perform the request, will manage to get the data from the targeted source.
Happy to work building a demo with bright data and work in achieving this.
Thanks
Dimitris
Ok guys, I’ll test and get back to you soon
1 Like
Zyte API is a bit too complex for me. I tried ScrapingBee and it works, but I didn’t get the price of the page ALTECH - Radiateur eau chaude Altech horizontal 4 connexions Type21 Hauteur 700 Largeur 900 1385W | Cedeo
Here it’s empty
And when I try something else, I got RuntimeError [400]
You need more specific selector than just span. Perhaps an AI would be able to guide you through the setup.
However, to scrape more complex sites, some knowledge of frontend development is generally required.
Ok @Milan_Vasarhelyi I understand. What’s the best coding language and scenario modules to do that on Make?