Filtering HTTP Response_Need Help

Hello, I’m scraping my own web database with “Make A Request” Module.
I managed to get my data right.

However, I don’t know one silly thing.

That is, what filter to choose when I want my scenario to start if only existing data is not scraped already. Because my scenario is keep repeating itself over and over. I want to filter it right at the start that my operations wouldn’t be consumed.

I tried to include a filters, but with no luck. My scenario look like this:

Relevant info about the scenario:
HTTP Filter Problem.json (12.3 KB)
Model-1-output-bundle.json (15.1 KB)
Module-1-input-bundle.json (1.6 KB)

My HTTP module:


You’ll need to put a Google Sheets “Search Rows” somewhere so the scenario will “know” which rows you need to scrape. Otherwise how would you know which rows you need to filter out?

2 Likes

But if I start in a website that has data I want to scrape?
The problem is that my scenario keeps repeating itself and scrapes the same information from the website.

I need to scrape all rows. Technically, I want to filter not rows, but an information on my website. I want to think scrape information on a website ONLY if it wasn’t scraped already. That’s the problem.

@vendy @Michaela can anyone please answer this question.