GET HTTP Request - wait to load the page

How To
1

What are you trying to achieve?

Get all the HTML data from a page that take some seconds to load some information.

Steps taken so far

Hey guys, there is a page that takes some seconds to load the data and when I get the output data from the GET http request module and don´t receive it.
How can I get the html code after waiting a few seconds?

thanks

2

Welcome @Gabriel4; it sounds like just using a simple HTTP request module won’t do it. As such, you’ll need to look into web scraping tools which would help with the page load delays and processing JavaScript.

I’ve the most familiarity with https://ujeebu.com/ since there’s separate extract and scrape paths that speed up my content collection. Alternatively, I’ve been thinking of ScrapeNinja for when Ujeebu finally fails me.

3

Welcome to the Make community!

For web scraping, some apps you can use are ScrapingBee and ScrapeNinja to get content from the page.

I’ve used ScrapeNinja, and you can use jQuery-like selectors in the extractor function.

ScrapeNinja also can run the page in a real web-browser, loading all the content and running the page load scripts so it closely simulates what you see, as opposed to just the raw page HTML fetched from the HTTP module.

If you want an example, take a look at Grab data from page and url - #5 by samliew

Hope this helps! Let me know if there are any further questions or issues.

— @samliew

P.S.: Did you know, the concepts of about 70% of questions asked on this forum are already covered in the Make Academy. Investing some effort into it will save you lots of time and frustration using Make later!