Automate Website actions

Hi
I’m trying to use Make.com for the first time, my work involves a lot of logging in to random websites , performing very similar tasks and closing out.

I’ve researched a little bit and I’m having a hard time finding documentation. I understand some websites use cloudflare and other means to prevent automation but what if I wanna try making my automation, what steps should I perform ?

From what I gather, I need to translate my website actions into specific HTTP requests that I can put in the HTTP module, but i’ve never done that before and I’m curious as to how to get started ?

Thanks a lot

1 Like

Hi Hardisk, can you please specify which websites for example and what exactly you are trying to do there

2 Likes

Welcome to the Make community!

For web scraping, some apps you can use are ScrapingBee and ScrapeNinja to get content from the page.

I’ve used ScrapeNinja, and you can use jQuery-like selectors there in the extractor function.

ScrapeNinja also can run the page in a web-browser using the “Scrape (Real Browser)” module so it closely emulates what users see, as opposed to just the raw page HTML fetched from the HTTP module.

If you want an example, take a look at Grab data from page and url - #5 by samliew

One thing to note, is that sometimes the scraping services do more than just fetch the resulting page - they might also do error handling, or retries, or bypass firewall (Cloudflare), etc., so it can be more reliable than just the standard HTTP module.

If you read about ScrapeNinja on their website,

ScrapeNinja handles headless browsers, proxies, timeouts, retries, and helps with data extraction, so you can just get the data in JSON.

Rotating Proxies, Unlimited Volume.
ScrapeNinja uses high quality proxies under the hood, so you don’t spend time managing and monitoring them, and it won’t break your bank.

Chrome Fingerprint Emulation
Getting blocked when scraping using CURL or PHP? ScrapeNinja emulates real Chrome request, without Puppeteer overhead (but JS evaluation is available, too!).

Smart Retries
ScrapeNinja retries based on timeout, response code and page text.

JS Evaluation
Choose if you want to evaluate JS (launch real headless browser) or perform raw network request (to get best performance and lowest latency possible). Experiment and switch between 2 network engines of ScrapeNinja on the fly.

Extract pure data from HTML. Faster.
Rapidly write & test simple JS snippets in online editor, to extract JSON from arbitrary HTML, via JS Extractors feature.

Build and test scrapers right in your browser.
Rapidly build and test web scraper in online builder, without writing a single line of code. Copy&paste generated web scraper code to your environment when you are done.

Links

Here are some useful links and guides you can use to learn more on how to use the Make platform, apps, and app modules. I found these useful when I was learning Make, and hope they might benefit you too —

General

Help Center Basics

Articles & Videos

2 Likes

Thanks for the response, i’ll try that. But it’s a lot to learn ! Would be great to see a better nocode tool to automate interacting with websites