It's always the JSON Parser

Hey guys, I ran into another error parsing data from ChatGPT to the JSON Parser. The error code generated is “Source is not valid JSON.” But, the JSON inside the ChatGPT module is correct. I’ve run it through Claude, ChatGPT, and https://jsonformatter.org/ yet the issue persists. The JSON is extensive and long so posting it here would be a lot.

I’ve also tried using the get variable transformer from the Tools module and the data exists, and even under the error code I get results (please refer to the screenshot):

Any help would be greatly appreciated!

I forgot to add, the goal here is to parse the JSON from the ChatGPT module to then feed to another ChatGPT module that completes “Timelines”. If you have any questions or can assist in anyway, I’d really appreciate it!

Hey Diego,

It seems that there are not enough tokens for this chatGPT response. Look at the end of the JSON - the content is cropped and not closed.

3 Likes

Thank you for your reply Mariana,

even when I increase the max tokens up to 2300, the problem still persists.

But is the generated JSON complete as a result?

Have you tried decreasing the max tokens? FOr testing purposes, make it very short so you can see if your process works. The max tokens, if I remember correctly, tells ChatGPT to limit its response to less thant that number of tokens.

Thank you Mariana and L Duperval for taking the time to answer. I made this quick video pertaining to the issue at hand - maybe it’ll provide more context :slight_smile:

HI Diego,

Can you share the input and output bundles? Also, in your JSON parser, I don’t see what it’s trying to use to parse the data. So maybe the input is wrong. Although, from your video, you seem to say that you’ve validated the JSON, so maybe it’s the configuration of the JSON parser.

And I’m interested because I’m in the process of building something similar to what you’re doing.

L

glad to hear you’re building something similar!

What I’m pointing at the JSON parser is the content dynamic variable under the message tab. I’ve also tried the result dynamic variable for the JSON parser to parse through it but the error persists. What’s odd is that despite the error code, if I inject a get variable transformer, there is an output but it can’t be identified.

I think the problem is that the JSON isn’t validated? Please refer to screenshot here:

Hmmm…

Did you see this:

It looks a lot like your problem.

Another one I saw was this:

Where the data is already parsed so you don’t need the JSON parser.

Another option is to create a JSON structure using the output from ChatGPT and using the Generate option in the JSON parser.

L

As you have shown again chatGPT used more than 2300 tokens and it means that answer is incomplete.
image
Give an additional rule to the system prompt: “use no more than n tokens for answer”.
It solved the problem for me:

I could be wrong but I think the max tokens is for the output tokens. Total tokens is a combination of input and output.

From Perplexity:

When discussing the number of tokens in the completion for the ChatGPT API, it specifically refers to the output tokens, not the total tokens (input + output).

Here’s a breakdown of how tokens are counted in the ChatGPT API:

  1. Prompt tokens: These are the tokens used in the input messages, including system, user, and assistant messages[1][3].

  2. Completion tokens: These are the tokens generated by the model in its response[1][3].

  3. Total tokens: This is the sum of prompt tokens and completion tokens[1][3].

The max_tokens parameter in the API request specifically limits the number of completion tokens (output tokens) that the model will generate[2][4]. It does not include the prompt tokens.

For example, if you set max_tokens to 100, the model will generate up to 100 tokens in its response, regardless of how many tokens were in the prompt. However, both prompt and completion tokens count towards the model’s maximum context length[2][4].

It’s important to note that the actual number of completion tokens may be less than the max_tokens value if the model completes its response before reaching the limit[4].

When using the streaming option, the API doesn’t provide token usage information in the streamed response[3]. To calculate token usage in this case, you would need to use a tokenizer (like tiktoken) to count the tokens in both the prompt and the generated response[3].

Citations:
[1] ChatGPT api maximum token - ChatGPT - OpenAI Developer Forum
[2] What is the maximum response length (output tokens) for each GPT model? - API - OpenAI Developer Forum
[3] Reddit - Dive into anything
[4] Can I set max_tokens for chatgpt turbo? - API - OpenAI Developer Forum
[5] openai api - How to calculate the token of the entire ChatGPT conversation? - Stack Overflow

Hey L and Mariana, thank you both. I ended up using both suggestions and the outcome still remains that when I feed “Content” from ChatGPT, the PandaDoc values don’t come up with anything.

The first link you provided L was what I tried first but the JSON parser didn’t pass the data unfortunately. The second option was able to generate the screenshot value but, the output from ChatGPT didn’t appear in the PandaDoc proposal.

Mariana, I did change the system prompt to have it up to 250 tokens. But, unfortunately whether I add a JSON parser in the next step or not, the data isn’t being recognized in the next module.

This ChatGPT module is meant to generate two components for the actual proposal and then generate an AI-generated timeline. But, in testing the output of this JSON parser, there’s nothing that can pass the data to the next module :smiling_face_with_tear:.


Hmmm, weird.

If you’re OK sharing the content of your prompt and your JSON, I can try to reproduce the problem and see if I can find a solution.

At this point, without trying the process and understanding the internal workings, I’m not sure I can offer anything more.

L

Thank you L! I’ll provide you the prompt and JSON as follows:

System prompt: You are a world-class, intelligent, sales assistant. Use no more than 250 tokens for your answer.

User prompt: We’re an AI Infrastructure agency that builds outreach systems, CRM systems, project management systems, no-code systems, and integrations.

Below is a loose scope. Turn that scope into a high quality proposal in JSON.

Our clients are enterprise, so write in a spartan, no-frills tone that implies intelligence.

User:

{
“contactInfo”: {
“firstName”: “{{3.mappable_answers.36ded7f4-1fc3-4651-b2f5-29fa1b378be7}}”,
“lastName”: “{{3.mappable_answers.fa5741e7-6ab1-41e0-a35b-3f599401be3b}}”,
“phoneNumber”: “{{3.mappable_answers.1c1fb7ba-a8ba-4378-a5cb-396095e6fe3f}}”,
“email”: “{{3.mappable_answers.b97bcceb-6486-4f47-92d9-96848570574d}}”,
“company”: “{{3.mappable_answers.1d578975-af3f-4dd2-b663-c7234a1d1e93}}”
},
“businessOverview”: {
“description”: “{{3.mappable_answers.46c3ded1-7641-41a0-9bc0-eac4e9c2b174}}.”,
“primaryProductService”: “{{3.mappable_answers.b60ba65f-f5fa-4d70-91e8-d485f1e4c581}}”,
“yearsInBusiness”: “{{3.mappable_answers.c91bae46-1104-4965-b4a2-c5abffd103d3}}”,
“teamSize”: “{{3.mappable_answers.9fd74efd-4040-4670-8efe-91b73575b9cc}}”,
“uniqueSellingProposition”: “{{3.mappable_answers.b60ba65f-f5fa-4d70-91e8-d485f1e4c581}}”,
“mainCompetitors”: [
“{{3.mappable_answers.dfb3fcb2-4fc1-4cb0-8766-f558b9774968}}”
]
},
“problemAnalysis”: {
“problemDescription”: “{{3.mappable_answers.0399385e-3af6-4461-bf12-26501f7c3494}}”,
“complexityRating”: “{{3.mappable_answers.a2a61a63-a347-4339-b2d2-f0d928605b10}}”,
“hoursSpentWeekly”: “{{3.mappable_answers.9bd96b97-6882-4736-8729-f9ab76028c90}}”,
“financialImpact”: “{{3.mappable_answers.04294245-54a8-4c39-9405-61d5cf9a9ef9}}”,
“leadConversionRate”: “{{3.mappable_answers.ccf01603-534b-4e91-ae55-124e6a7471b9}}”,
“previousAttempts”: “{{3.mappable_answers.3d810ccf-7a66-499a-9b7f-2df0790883a5}}”,
“industryTerms”: [
“{{3.mappable_answers.12d81298-6693-4727-9253-45ed55ec23f9}}”
]
},
“proposedSolution”: {
“description”: “{{3.mappable_answers.9a3b189c-26b7-434d-a711-7d235d14e23c}}”,
“platformsTools”: [
“{{3.mappable_answers.8dea8e47-b0dc-41f7-a726-ea37ab52847a}}”
],
“projectDuration”: “{{3.mappable_answers.8a19b3ef-2fad-4fe3-9974-5a35f41e4ffc}}”,
“cost”: {
“solutionCost”: “{{3.mappable_answers.6f28c2c3-4a10-4323-a096-1913b3d8cb2d}}”,
“additionalPlatformCosts”: “{{3.mappable_answers.52367371-d707-4de2-a2aa-6b5f59bd784a}}”
}
}
}

Assistant: {
“ClientProblemArea”: “Flatiron Search Partners, a leading executive recruiting firm with 15 years of experience in the high-end technology sector, is grappling with a critical operational challenge. Their current system of managing leads and projects via a massive Google Sheet is severely hampering their efficiency and scalability. This problem, rated 4 out of 5 in complexity, is consuming approximately 30 hours per week of valuable time across their 25-member team. The financial impact is staggering, estimated at $500,000 annually in lost productivity and missed opportunities. Their current lead conversion rate stands at a mere 15%, well below industry standards. This inefficiency is not only costing time and money but also undermining Flatiron’s unique selling proposition of having an unparalleled network in Silicon Valley and deep understanding of tech industry trends. Previous attempts to address this issue, including the use of multiple spreadsheets and a basic CRM tool, have fallen short due to lack of integration. In the competitive landscape where Flatiron faces rivals like TechTalent Recruiters and Valley Headhunters Inc., this system problem is severely limiting their ability to optimize their ‘time-to-fill’ metrics and effectively manage their candidate pipeline. Without addressing this issue, Flatiron risks losing its edge in the dynamic tech recruitment sector, potentially eroding their market position and stunting their growth trajectory.”,
“ClientSolution”: “To address Flatiron Search Partners’ challenges, we propose a comprehensive solution that leverages cutting-edge technology and industry best practices. The core of this solution is the migration to a dedicated CRM and project management platform, specifically Monday.com, complemented by custom integrations with Zapier and LinkedIn Recruiter. This solution is designed to streamline Flatiron’s entire recruitment process, from lead management to candidate placement. By implementing this system, we aim to reduce the 30 hours per week currently spent on manual data entry and management by at least 70%. The Monday.com platform will serve as a centralized hub for all client and candidate information, enabling real-time collaboration and providing a clear overview of the recruitment pipeline. Zapier integrations will automate routine tasks, such as updating candidate statuses and sending follow-up emails, further enhancing efficiency. The LinkedIn Recruiter integration will allow for seamless import of potential candidates, expanding Flatiron’s talent pool and improving the quality of matches. This solution is expected to significantly boost the current 15% lead conversion rate, potentially doubling it within the first six months of implementation. The project will take approximately 8 weeks to complete, ensuring a thorough and careful integration with Flatiron’s existing workflows. The investment required is $75,000 for the solution implementation, with an additional $15,000 annually for platform subscriptions. While this may seem substantial, it pales in comparison to the estimated $500,000 annual loss due to current inefficiencies. Moreover, this solution will enhance Flatiron’s unique selling proposition by enabling them to leverage their industry knowledge and network more effectively, setting them apart from competitors like TechTalent Recruiters and Valley Headhunters Inc. By streamlining operations and providing data-driven insights, this solution will not only address Flatiron’s immediate operational challenges but also position them for sustainable growth in the competitive tech recruitment sector.”
}

User:
{
“instruction”: “Based on the provided information, create high-quality, personalized [Client.ProblemArea] and [Client.Solution] sections for a business proposal. Follow these guidelines:\n1. For [Client.ProblemArea] (250-350 tokens):\n - Clearly state the client’s problem, emphasizing its complexity.\n - Quantify the problem’s impact using financial and time-based metrics.\n - Reference industry-specific terms and previous solution attempts.\n - Explain how the problem affects the client’s market position and goals.\n2. For [Client.Solution] (300-400 tokens):\n - Outline the proposed solution, including specific platforms and tools.\n - Explain how the solution addresses each aspect of the problem.\n - Provide expected outcomes and improvements.\n - Include project duration and cost breakdown.\n3. Incorporate industry-specific language and metrics where relevant.\n4. Personalize the content by referencing specific details about the client’s business.\n5. Maintain a spartan, no-frills tone that implies intelligence, suitable for enterprise clients.\n6. Structure each section to start with a clear statement, explain current situation or proposed actions, and outline consequences or benefits.\n7. End each section with a statement tying the problem or solution to broader business goals or long-term consequences”
}

Assistant:
{
“ClientProblemArea”: “Generated problem area analysis will appear here”,
“ClientSolution”: “Generated solution description will appear here”
}

Thanks. I’ll take a look at this later tonight (NA Eastern time) and see what I can come up with, if nobody else solves this for you beforehand.

L

Hi,

It’s the double quotes, they’re the wrong type that’s being used.

I put the first JSON you provided into JSON Parser Online to parse JSON

When I swap the “ for " (notice they look different) the line is successfully parsed.

Perhaps you can tell ChatGPT to use " when it’s forming the reply ?

Hey CJ, thank you for this. What’s funny is that I identified the " ‘’'json " before the actual JSON string in the output and just through dumb testing I guess, I put in the system prompt to remove that string above in the beginning and ending of the JSON string. Now, it’s fixed. Lmao :smiling_face_with_tear:

2 Likes

Hi,

Glad it’s solved :slight_smile:

L, thank you so much for all your help! I was able to override the problem by adding to the system prompt “remove the ‘’'json from the beginning and ending of the JSON output”. Super thankful for all your help!!

2 Likes