Hey Aarti,
if you are using an external AI it will cost tokens yes, which are based on prompt and response lengths for the most part. So the more complex the prompt and richer the output, the more tokens it will consume. Once you deploy enough of them you will start to get a feel for at least a rough estimate of how many tokens different prompts will burn through in a month, but I don’t think there is a good way to get an exact estimation right of the bat. And yeah, this is a big pain point.
While it works essentially the same way, using Make’s built in AI will consume from the account credits instead of additional tokens paid to a third party. So for me, its much easier to predict the costs with it, as you can also track the tools it uses as well and actually see the amount of credits each run consumes.
And if you are using your own model, it very much depends if you are hosting it locally (and thus only paying your internet bill, but being limited by what your ISP can provide) or if its cloud hosted. I think the cloud hosted model is the most unpredictable regarding bills at the end of the month though so I don’t recommend this.
Also for a Make hosted chatbot to cost 10k a month or upwards? Don’t know how many people do you plan on using it to get even close to that sum.