If you've ever asked ChatGPT or Claude to generate JSON, you've probably hit this: you paste the output into your code, and it throws a parse error. The JSON looks right, but it's subtly broken.
This isn't a rare edge case. LLMs produce invalid JSON frequently, especially for large or nested structures. Let's look at why it happens, how to fix each error type, and how to automate the repair process.
Why LLMs produce broken JSON
LLMs generate text token by token. They don't have a JSON validator running internally. That means they can easily produce output that looks like JSON but violates the spec in small ways. The bigger the JSON structure you request, the more likely something breaks.
Here are the most common errors.
Error 1: Trailing commas
This is the single most frequent issue. The LLM adds a comma after the last item in an array or object.
{
"name": "Alice",
"age": 30,
"hobbies": ["reading", "hiking",]
}
The fix: Remove the trailing comma before ] or }. In a regex:
// Remove trailing commas before ] or }
const fixed = broken.replace(/,\s*([\]}])/g, '$1');
Why it happens: Many programming languages (JavaScript, Python, Rust) allow trailing commas. JSON does not. The LLM's training data includes far more code than JSON specs.
Error 2: Single quotes instead of double quotes
{
'name': 'Alice',
'age': 30
}
JSON requires double quotes for all strings and keys. Single quotes are invalid.
The fix:
// Naive replacement — works for simple cases
const fixed = broken.replace(/'/g, '"');
Be careful with this one. If your string values contain apostrophes (it's), a simple replacement will break things further. A smarter approach:
// Replace single-quoted keys and values, preserving apostrophes in text
const fixed = broken.replace(/'([^'\\](?:\\.[^'\\])*)'/g, '"$1"');
Error 3: Unquoted keys
{
name: "Alice",
age: 30
}
This is valid JavaScript but not valid JSON. Every key must be a double-quoted string.
The fix:
const fixed = broken.replace(/(\{|,)\s(\w+)\s:/g, '$1"$2":');
Error 4: Comments in JSON
{
"name": "Alice", // user's name
"age": 30 / years old /
}
JSON does not support comments of any kind. LLMs add them because they're trained on commented code.
The fix:
const fixed = broken
.replace(/\/\/.*$/gm, '') // remove // comments
.replace(/\/\[\s\S]?\\//g, ''); // remove / */ comments
Error 5: Truncated output
Sometimes the LLM hits its output token limit mid-JSON:
{
"users":
{"name": "Alice", "age": 30},
{"name": "Bob", "age": 2
This is the hardest to fix automatically because you need to infer the missing structure. Manual options: close the open braces and brackets, or re-prompt the LLM asking it to continue.
Error 6: Markdown code fences
The LLM wraps its output in triple backticks:
json
{"name": "Alice"}
The fix:
const fixed = broken.replace(/^$/gm, '');
Fixing JSON programmatically with json_repair
For Python users, the
json_repair library handles most of these issues automatically:
bash
pip install json-repair
python
from json_repair import repair_jsonbroken = "{'name': 'Alice', 'age': 30,}"
result = repair_json(broken)
print(result)
{"name": "Alice", "age": 30}
It handles trailing commas, single quotes, unquoted keys, comments, and more. It's deterministic (no LLM calls to fix LLM output), and it works offline.For JavaScript, the
jsonrepair npm package does the same thing:
bash
npm install jsonrepair
javascript
import { jsonrepair } from 'jsonrepair';const fixed = jsonrepair("{'name': 'Alice',}");
Quick online fix
If you just need to paste some broken JSON and get valid output, [jsonshield.com/tools/json-fixer runs the repair client-side in your browser. Paste, fix, copy. No data leaves your machine.
Preventing broken JSON in the first place
A few prompting strategies that reduce (but don't eliminate) errors:
- Ask for small chunks. Instead of one giant JSON blob, request it in parts.
- Provide a schema. Show the LLM the exact structure you want with an example.
- Use structured output APIs. OpenAI's function calling and Anthropic's tool use both enforce valid JSON through constrained decoding. This is the most reliable approach.
- Validate in your pipeline. Always wrap
JSON.parse() in a try/catch, and have a repair step ready.
javascript
function safeParse(text) {
try {
return JSON.parse(text);
} catch {
const { jsonrepair } = require('jsonrepair');
return JSON.parse(jsonrepair(text));
}
}
`Structured output is the real fix
If you're building an application that relies on LLM-generated JSON, don't rely on free-text output. Use the structured output features that model providers offer:
- OpenAI:
These force the model to produce valid JSON by constraining the token generation at each step. The output always parses correctly.
Wrapping up
Broken JSON from LLMs is a "when, not if" problem. The good news is that the errors are predictable and fixable. For one-off fixes, a quick regex or an online tool works fine. For production code, use a repair library as a fallback and structured output APIs as your primary approach.