I’m pulling in a bunch of rows from Airtable to Make that I will analyse with LLM modules.
In some cases the rows might exceed the LLMs context window. Is there an easy way to count the tokens inside Make and cut them off at, say, 50k? So the input would be all of the rows from Airtable and the output would be the rows but cut off at 50k tokens.
Thank you!
Welcome to the Make community!
I have a private module that can estimate the number of tokens in a text input.
You can install it using this link: app-invitation/be1ae21f84e234e707093a0ae1658770 (don’t share this link with others, it’s top-secret!)
I would say it’s pretty close enough to the “official” result.
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.
Very cool, thank you so much! 
A bit offtopic, but let’s say I have text stored in a “get multiple variables” module, and I’ve calculated its token count with your module. If the token count exceeds a specific limit (eg. 50,000 tokens), what’s the best way to truncate the text to match that limit? Using the ‘slice’ function?
Good question, please create a new topic for this!
Just an update, here is a screenshot of the current modules in my Make custom app, which is currently completely free to use.
If you find any issues with any of my modules, or have new module ideas or requests, I would appreciate it if you can message me with any errors you find.
You can use the private messaging feature to directly reach out to other forum members. To do this, go to your profile, and click on the “New Message” button:
Hope this helps! Let me know if there are any further questions or issues.
— @samliew
P.S.: Investing some effort into the Make Academy will save you lots of time and frustration using Make.
