So I’ve got 2 Claude API call modules setup in Make to run 2 separate prompts. I’m using the following body for both API call modules:
{
“model”: “claude-3-5-sonnet-20241022”,
“max_tokens”: 4096,
“system”: [
{
“type”: “text”,
“text”: “You are a content analysis expert.” },
{
“type”: “text”,
“text”: {{TRANSCRIPTS_GO_HERE}},
“cache_control”: { “type”: “ephemeral” }
}
],
“messages”: [
{
“role”: “user”,
“content”: {{PROMPT_GOES_HERE}}
}
]
}
For context, these prompts analyse the video transcripts.
I ran both prompts and the caching seemed to work:
For prompt 1 I got:
| input_tokens | 685 |
|---|---|
| cache_creation_input_tokens | 5411 |
| cache_read_input_tokens | 0 |
| output_tokens | 883 |
For prompt 2 I got:
| input_tokens | 577 |
|---|---|
| cache_creation_input_tokens | 0 |
| cache_read_input_tokens | 5411 |
| output_tokens | 690 |
However, when I tried changing the text for the systems prompt, ie. “You are a content analysis expert.” for one of the prompts, then the token count for “cache_creation_input_tokens” mis-matched and I wasn’t able to successfully cache the transcript anymore.
How should I structure the body so that I could change the “text”: “You are a content analysis expert.” for both prompts, while still caching the transcript? Ie. “text”: {{TRANSCRIPTS_GO_HERE}},
I tried asking o1 this, and it gave me a few outputs, but none of them worked.
Please advise. Thanks!