How to Scrape a Website that Requires Login?

Can I use http modules to Scrape a Website that Requires Login? (I mean this website don’t have api or oauth, I only have username and password to login that website)

I know I can use scrape service from third party. But they are expensive, so I want to use http module to scrape data.

Please help me, thanks.

1 Like

I know this is late.

this is how you login to a site with make (the 3rd post here.)
you can technically parse the HTMl using make, but honestly would recomend using apify webscraper.

I run maybe 2500 scrapes a month with it and my monthly cost is between $30 and $50. half of that cost is from two specific monthly scrapes of video data which is quite a bit more expensive to run. my average scraper run cost is maybe $0.02 or less when grabbing table or html based data.