Author(s): Sourav Ghosh

Originally published on Towards AI.

Is JSON Finally Getting a Token-Efficient Alternative for LLMs?

For years, JSON has been the default language for APIs, integrations, configuration files, event payloads, and all other types of application-to-application communications. It is an easy language to understand, it is very robust and developers can easily exploit it. But when we transition from traditional software systems to Large Language Model applications, we start to see how JSON comes with an invisible price tag. LLMs do not process JSON the way that applications do. They handle it as tokens.

The article explains why JSON becomes token-expensive for LLMs—repeated keys, syntax, and nested structure consume context window and increase cost—then introduces TOON (Token-Oriented Object Notation) as a more token-efficient, prompt-friendly way to represent structured data while preserving the same underlying data model (objects, arrays, strings, numbers, booleans, null). It shows a before/after example converting JSON arrays of records into TOON where field names are declared once, values are arranged in rows, and structure remains readable for the model. The piece argues TOON is especially valuable at the LLM boundary when payloads share a uniform schema with repeated records (common in RAG retrieval results, agent tool outputs, and agent memory), and it provides enterprise scenarios plus code/prompt patterns illustrating how to use TOON as LLM input while keeping JSON for validated outputs. Finally, it outlines best practices and cautions: don’t replace JSON everywhere, use TOON only where it fits (and validate outputs), benchmark against JSON, consider tooling/model reliability and escaping edge cases, and treat TOON as an optimization layer for context representation rather than an enterprise contract substitute.

Read the full blog for free on Medium.

Published via Towards AI