How to avoid Captcha when using HTTP module

Questions & Answers
1

I am using the HTTP module, trying to do a GET request to this URL:

https://canlii.org/en/on/onarb/doc/2023/2023canlii3033/2023canlii3033.html

When you manually copy/paste this URL into a browser, it works fine. The problem arises when I use the HTTP module which triggers a Captcha and HTTP 429 Too Many Requests response as shown below.

My question is: What are some strategies that I can use to avoid triggering the Captcha so that I can GET the page?

canlii captcha
canlii captcha1102×1715 126 KB

2

The best way to avoid is to add a SLEEP module with a 15-20 seconds delay in each request.

1 Like
3

Thanks for your suggestion Manish, but after implementing the SLEEP module, I am still receiving the same Captcha page and 429 error. Are you able GET the contents of the page using an HTTP request?