I need to scrape the URLs from this page
The table is within the DIV with ID “partnerprofiles”. If I read it in my browser, the table is loaded as I scroll down. If I use an HTTP Get module, “partnerprofiles” is empty.
For this specific case, I can just copy and paste the source of the page. But in general, when I find similar pages, how do I scrape them?