Issues with filtering two sources

[n_c]

I’ve used 19,000+ operations trying to get this to work:

Documentation: HTTP Basic Auth Request for Shelter Manager Integration with Webflow CMS

Shelter Manager Documentation

Objective:

The goal is to integrate a non-profit animal shelter’s database, managed by Shelter Manager, with their website built with Webflow CMS. This involves syncing the animals’ data (dogs and cats) between the two platforms.

The intended operations include:

1. Adding new animal entries from Shelter Manager to the Webflow CMS collection.
2. Updating existing animal entries on Webflow CMS if changes occur in the Shelter Manager database.
3. Deleting animal entries from the Webflow CMS collection if they are removed from the Shelter Manager database.

These operations should take place 1-2 times a day using Make.com’s services.

Current Situation:

Currently, the scenario is only able to add new items to the specified Webflow CMS collection. Each run of the scenario results in new CMS items being added from the first iterated bundle by the Iterator module which is iterating data from the HTTP Make a Basic Auth Request module. This leads to duplication of items in the CMS collection.

Proposed Strategy:

It was initially thought that by attaching unique IDs (20. ID:) from the Iterator module to a field (Pet ID:) in each Webflow CMS Item, the data could be filtered to go to one of three Webflow operations: Create, Update, or Delete.

1. Create an Item: If there is no existing Webflow CMS Item with a matching value in the Pet ID field, a new item would be created.
2. Update an Item: If a Webflow CMS Item with a matching ID in the Pet ID field exists and the ID still exists in the iterated data from the HTTP Make a Basic Auth Request module, the existing item would be updated.
3. Delete an Item: If a Webflow CMS Item contains a Pet ID value that no longer exists in the iterated data from the HTTP Make a Basic Auth Request module, that item would be deleted.

Problem:

Despite multiple attempts with various filters, tools, and arrangements, the Update and Delete operations have not been successful. Also, the system hasn’t been able to prevent duplication of Webflow CMS.

Also, the scenario only deals with ~105 pets coming from the HTTP Get but burns operations like crazy.

[IOA_Harman]

Hi, it seems like your close to solving this, here’s my advice.

Your proposed strategy is correct, the implementation would be something along the lines of:

Step 1: Search all the data existing in Webflow CMS, and store it in an aggregator.

Step 2: Iterate through the output from the initial GET HTTP request, so that we can go through all the data, not only the first bundle. If you could provide screenshots of how this is currently being handled, I may be able to help.

Step 3: Compare the data in the current iteration to the data stored in our aggregator, you will likely need to use some map() and get() functions to accomplish this.

-IF item exists in Webflow → Check if it needs to be updated → either update or do nothing
-IF item does not exist → Create Item

-IF item exists in Webflow & Not in our DB → Delete Item in Webflow

Here is a screenshot that I believe will be similar to your end result:
Note that I named the module after what I expect your step to be.
You would need to adjust all the filters I labelled to do handle the steps above.

Harman - IOA Digital