Using ScrapingBee and Getting runtime error 400 for my JSON used, not sure why, it should work

Questions & Answers
,
1

Im using ScrapingBee and I input this JSOn and it works fine and return the 2 expected items from my personal website. But when I add 3 more CSS selectors to scrape, I get a runtime error. Logic is the same and the CSS selectors are on the page so they should find them also, but I get runtime error 400.

this works fine in MAKE AND ScrapingBEE test console:

{
“zodiac” : “.yellow-sign”,
“entry-date” : “.yellow-date”

}

but this works fine in ScrapingBee test dashboard BUTgives runtime 400 error in MAKE :

{
“zodiac” : “.yellow-sign”,
“entry-date” : “.yellow-date”,
“next-date” : “.next-date”,
“next-sign” : “.next-sign”,
“next-time” : “.next-time”
}

the ONLY difference is that the first 2 items on the page are written in php and from a mysql query without a loop, and the other 3 are in a while loop as they are written, but as far as the web page and scraping, its just elements on a page in a CSS selector to scrape right? So shouldnt make a difference, excet since its a loop there are more than one of each. Many rowas actually, but I just need the first one, but I tried doing :first-of-type but Scraping bee does not recognize those kind of selectors. Like

{
“zodiac” : “.yellow-sign”,
“entry-date” : “.yellow-date”,
“next-date” : “.next-date:first-of-type”,
“next-sign” : “.next-sign:first-of-type”,
“next-time” : “.next-time:first-of-type”
}

Anyone good at scraping who knows how to get around this or what im doing?

3

Welcome to the Make community!

Does the runtime error state which field is invalid?

1.

Could you please share screenshots of the module fields and filters in question? It would really help other community members to see what you’re looking at.

2.

You can also export the scenario blueprint file to allow others to replicate the issue. At the bottom of the scenario editor, you can click on the three dots to find the Export Blueprint menu item.

Screenshot_2023-08-24_230826

Uploading it here will look like this:

blueprint.json (12.3 KB)

3.

If possible, could you also please provide the input/output bundles of the modules by running the scenario, then clicking the bubble on the top-right of each module, taking screenshots of, and copying the contents into this discussion thread:

Screenshot_2023-08-29_100800

This will allow others to better assist you. Thanks!

2 Likes
4

In answer to #1. there was no specific field or anything mentioned, just runtime error 400 and some very geneeric lingo.

BUT the good news is it just started working all of a sudden after a day of not working. No idea why it started working as I didnt do anything. If I had to guess it seems like MAKE was caching the sites OLD version prior to me making some changes to the site being scraped, as it was my own personal site and so I made it easier to scrape :slight_smile:

So it just stopped giving the error. which it should not have as I said it worked fine on the ScrapingBee scrape builder page.

So we can close this

2 Likes
5

Hey @Norman_Love and welcome to the community :wave:

Great to hear that your scenario is working now! :smile:

Also thank you very much for sharing the updates and marking your response as a solution. This way it’s easier for other Makers to look for solutions. :pray:

Keep up the great work!

1 Like