What are you trying to achieve?
I am trying to scrap url content, and sending html output to text make parser app. But i have some issues (You need to enable JavaScript to run this app) see the attached image.
Can some on help me to solve this.
I know it doesnât answer your question, but ever since all of the AI modules, I use AI modules exclusively for Parsing and Coding. Iâve always found that coming up with those REGEX expressions were just way too complex than it needs to be. Iâve also tried EVERY code module and with AI you can use plain English to solve your questions and most of the AI stuff is virtually free. Furthermore DeepSeek can output its Resukts as JSON.
Welcome to the Make community!
So you basically need to âvisitâ the site yourself to get the content. This is called Web Scraping.
Incomplete Scraping
Are you getting NO output from the Text Parser âHTML to Textâ module? This is because there is NO text content in the HTML! The entire page content you are scraping is hosted in a script tag, which is dynamically generated and placed onto the page using JavaScript when loaded and run on the userâs web browser on the client-side. Make is a server-side runtime environment, so using the HTTP modules, you get just the script tags, and those script tags are ignored by the Text Parser âHTML to Textâ module because it is NOT a HTML layout element.
Using the Make HTTP âMake a requestâ does NOT run any of those JavaScript scripts, so there is no content on the page other than a default message that tells you to enable JavaScript.
This is NOT a Make platform, or Text Parser, or Regular Expression issue/bug.
You CANNOT use normal scraping integrations like ScrapingBee or HTTP âMake a requestâ module to fetch this pageâs structure.
You will need to use ScrapeNinjaâs âScrape (Real browser)â module to emulate a real person visiting the site using a web browser, as client-side JavaScript needs to run to parse the JSON data in the script tags, and generate the page structure and content.
For more information and demo using ScrapeNinja, see Scraping Bee Integration Runtime Error 400
Web Scraping
For web scraping, a service you can use is ScrapeNinja to get content from the page.
ScrapeNinja allows you to use jQuery-like selectors to extract content from elements by using an extractor function. ScrapeNinja also can run the page in a real web-browser, loading all the content and running the page scripts so it closely simulates what you see, as opposed to just the raw page HTML fetched from the HTTP module, which fails as in your case.
If you want an example, take a look at Grab data from page and url - #5 by samliew
AI-powered âeasierâ method
You can also use AI-powered web scraping tools like Dumpling AI.
This is probably the easiest and quickest way to set-up, because all you need to do is to describe the content that you want, instead of inspecting the element to create selectors, or having to come up with regular expression patterns.
The plus-side of this is that such services combine BOTH fetching and extracting of the data in a single module (saving operations), and doing away with the lengthy setup from the other methods.
More information, other methods
For more information on the different methods of web scraping, see Overview of Different Web Scraping Techniques in Make đ
Hope this helps! Let me know if there are any further questions or issues.
â @samliew
P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.
I didnât know that
Thank you for your answer
However, I came across a video of Make using exactly the HTTP App and the text parser to scrape.
So I donât really understand why it doesnât work for me!
I said it is something to do with the website you are scraping. Notice your website is different from the video tutorial?
Hope this helps! Let me know if there are any further questions or issues.
â @samliew
P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.
2 Likes