# Need Help Filtering Duplicate Scraped Data in Automation Workflow **URL:** \ **Category:** Questions\ **Tags:** filters, airtable\ **Created:** [March 1, 2024, 8:40am UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149 "2024-03-01T08:40:34Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Javier\_Solans](https://avatars.discourse-cdn.com/v4/letter/j/96bed5/32.png) [@Javier\_Solans](https://community.make.com/u/Javier_Solans)\ **Post date:** [March 1, 2024, 8:40am UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/1 "2024-03-01T08:40:34Z")
Hello! I’m facing an issue with my automation workflow on [make.com](http://make.com). Here’s my setup: search airtable → text aggregator → scraper → iterator → create airtable record. The problem is, I want to avoid uploading duplicate scraped data to Airtable. I tried adding a ‘search airtable’ step, but I’m struggling to filter out existing data at the end. Can anyone suggest a solution to ensure that only new scraped data is uploaded to Airtable? Thanks! ![image](https://europe1.discourse-cdn.com/flex013/uploads/make/original/3X/1/f/1f6a1a380fe5d8b297bedb64a9aca89cd2e46a4f.png) ![image2](https://europe1.discourse-cdn.com/flex013/uploads/make/original/3X/8/6/8648336ec7668e6f61f8bdf7a199e7c6126f9b43.png) ![3](https://europe1.discourse-cdn.com/flex013/uploads/make/original/3X/a/8/a874f6a1353a2b8c7f938b5f19be2fc799f0a8cf.png) ---
**Author:** ![Javier\_Solans](https://avatars.discourse-cdn.com/v4/letter/j/96bed5/32.png) [@Javier\_Solans](https://community.make.com/u/Javier_Solans)\ **Post date:** [March 1, 2024, 9:51am UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/2 "2024-03-01T09:51:38Z")
I’ve been struggling with this for a long time already, hopefully someone can help 🙂 ---
**Author:** ![Dorian\_Ben\_Haim](https://dub1.discourse-cdn.com/flex013/user_avatar/community.make.com/dorian_ben_haim/32/27858_2.png) [@Dorian\_Ben\_Haim](https://community.make.com/u/Dorian_Ben_Haim)\ **Post date:** [March 1, 2024, 12:38pm UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/3 "2024-03-01T12:38:49Z")
Hi @Javier_Solans You can upsert records (update/insert) [Make Airtable upsert documentation](https://www.make.com/en/help/app/airtable#upsert-a-record) You can search for the record id with the search record module by looking for some unique param ---
**Author:** ![Javier\_Solans](https://avatars.discourse-cdn.com/v4/letter/j/96bed5/32.png) [@Javier\_Solans](https://community.make.com/u/Javier_Solans)\ **Post date:** [March 1, 2024, 12:57pm UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/4 "2024-03-01T12:57:10Z")
Thank you for the answer. And would it be possible to just insert the new ones and do not insert the ones that its name already exists in the table? Additionally, it would be less records created then less operations runned in [make.com](http://make.com) ---
**Author:** ![Dorian\_Ben\_Haim](https://dub1.discourse-cdn.com/flex013/user_avatar/community.make.com/dorian_ben_haim/32/27858_2.png) [@Dorian\_Ben\_Haim](https://community.make.com/u/Dorian_Ben_Haim)\ **Post date:** [March 1, 2024, 2:09pm UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/5 "2024-03-01T14:09:59Z")
Without a unique identifier it’s impossible But yo can use list records via “make an api call” module and then aggregate the data using filters in the between to remove the existing records and it will cost only 2 operations For more information regard the list records you can read in the [Airtable documentation](https://airtable.com/developers/web/api/list-records) ---
**Author:** ![Javier\_Solans](https://avatars.discourse-cdn.com/v4/letter/j/96bed5/32.png) [@Javier\_Solans](https://community.make.com/u/Javier_Solans)\ **Post date:** [March 1, 2024, 2:54pm UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/6 "2024-03-01T14:54:08Z")
Would you be up to have a chat? [LINK](https://calendly.com/winnersdaily/schedule-a-demo-with-javi-from-winnersdaily) ---
**Author:** ![Make\_Bot](https://dub1.discourse-cdn.com/flex013/user_avatar/community.make.com/make_bot/32/14661_2.png) [@Make\_Bot](https://community.make.com/u/Make_Bot)\ **Post date:** [May 30, 2024, 2:04pm UTC](https://community.make.com/t/need-help-filtering-duplicate-scraped-data-in-automation-workflow/29149/7 "2024-05-30T14:04:04Z")