Nice work. I do usually suggest for people to use a dedicated scraping API however, because that handles all of this and more specific specific issues.
Usually it’s worth it to look on the Apify Store for website specific actors, or check out more general actors such as Content Crawler