I am currently working on a project where my task is to design a scenario that will retrieve all product (meal) data from an API URL endpoint and load it onto a new website. The api url has total 10 pages right now. The API may have up to 200 products, with the potential for more or less in the future. The scenario will run every hour, and each product has a unique ID, SKU, name, description, and other attributes.
I am using Make’s data store to store the data from the API, and I have figured out how to add a new item or update the attributes of a current item in my data store. However, I am stuck on how to remove an item from the data store if it has been deleted from the API when my scenario runs.
Do you have any suggestions or approaches that I can use to solve this problem? Thank you very much!
One scenario will call api and check all products to create or update. While creating or updating you can use tag or date updated to make datastore updated that when the last product was updated.
When all products have been checked then you will be left with products which havent updated yet and those are the deleted products from api…
So the 2nd scenario will have “search datastore” module. Use filter in it to get products that are not updated yet. And the delete record to remove it from datastore.
Thank you very much! I like your ideas ^^
I have a follow up question, how you can use filter to get the products have not updated using “search datastore” module in scenario 2? As my understanding, scenarios 2 has to be scheduled after the scenario 1 finishes its run. Is this possible if I compare the hour to get the removed items?
If it is important to keep the datastore updated then I will suggest scheduling scenario in a way that it runs after 1st scenario. Otherwise schedule it for midnight.
Mark this solved if you don’t have any follow up question.