Did some more searching and it looks like there is no way to get what I want just from Aggregators.
Also found some methods or apps that others have created to help with Google Sheets batch updates.
This post by @make_expert describes a method for using replace functions to escape JSON characters.
The upside, it only takes 3 Ops from Iterator → Aggregator → Batch Update.
The downside, you have to manually map each column and use this complicated replace() function on any column you think will have a value that contains characters that may need to be escaped. On a good note, I completely forgot the replace function allows for regex, which is super helpful here.
Growwstacks[.]com has a Google Sheets Batch Update Custom App. You have to provide your info, they apparently respond with a link to install the app.
Msquare_Automation has a very similar custom app to Growwstacks, except it looks like it’s paid.
makeitfuture also has a custom app, also appears to be paid.
I guess these are basically all the current known solutions to batch update for Google Sheets, and maybe Excel 365 as well.