Can I use http modules to Scrape a Website that Requires Login? (I mean this website don’t have api or oauth, I only have username and password to login that website)
I know I can use scrape service from third party. But they are expensive, so I want to use http module to scrape data.
this is how you login to a site with make (the 3rd post here.)
you can technically parse the HTMl using make, but honestly would recomend using apify webscraper.
I run maybe 2500 scrapes a month with it and my monthly cost is between $30 and $50. half of that cost is from two specific monthly scrapes of video data which is quite a bit more expensive to run. my average scraper run cost is maybe $0.02 or less when grabbing table or html based data.