Working with webhook data…well it can be great because you get instant results.
It can also be bad, because if you don’t process that data when it’s sent by the webhook, it could be lost F O R E V E R !
Well, not forever, but it does require the use of Postman to retry / test with data that wasn’t fully processed.
That’s why I’ve made it one of my “Best Practices” to always use Data Store as a Webhook Data Buffer.
What is this gypsy magic you ask? Keep reading…
Below is a scenario where I watch QuickBooks Online for new events. When data is received from QBO, data is stored in the Data Store webhook buffer for future retrieval.
As you can see below, I’m saving the QBO Object ID, Type and Operation Type for the QBO asset. I’m also saving the QBO last modified time, current time and the execution log URL for this buffer record.
Note, I have “Overwrite Existing Record” set to Yes, because I don’t want to create any duplicate buffer records. This keeps everything unique.
Once the buffer record is saved, I use multiple HTTP modules to trigger the next scenario.
Next, you will have a companion scenario that will 1) start with a webhook, 2) search the data store buffer, then complete the steps that must be automated.
NOTE: Set your Data Store Search module to retrieve more than one (1) record if you want it to processed webhook data that has not been processed by your scenario. This harnesses the power of the webhook buffer. If a past webhook data package wasn’t processed, it can be attempted again when another bundles is sent through your scenarios!
Next, and this is important, add a Delete Data Store module at a place in your scenario where you no longer need your webhook buffer data. This ensures your data will not be processed a second time.
Here’s a summary:
Scenario A has a webhook module as the trigger (this webhook contains API service data payload), and an action that saves the webhook data into a DataStore record. Also present is an HTTP module that fires the Scenario B webhook.
Scenario B starts with a webhook, searches for the saved Data Store Buffer records, then performs the necessary scenario actions, then ends by deleting the Data Store Buffer Record so it isn’t processed again.
If you get stuck, reach out and myself or a member of my team can help!