TOON Converter Documentation

Token-Optimized Object Notation (TOON) v2.0 - The most comprehensive Python library for TOON format, featuring 100% spec compliance, 10 framework integrations, and production-ready tools for reducing LLM token usage by 30-60%.

Why Use TOON Converter?

Real Benefits for Your LLM Applications

Save Money: 30-60% token reduction = $1000/mo → $400/mo on API costs
Faster Processing: Smaller payloads = faster responses (200ms → 80ms average latency)
Better Context: More data in same token limit (Fit 10 docs instead of 6 in context)
Works Everywhere: 10 framework integrations: LangChain, Pandas, FastAPI, SQLAlchemy, MCP
Easy to Use: 2 lines of code to get started: import toonverter as toon; toon.encode(data)
Production Ready: Battle-tested, type-safe (563 tests, 81% coverage)
Smart Optimization: Auto-detects tabular data and uses compact table format
Format Flexibility: Convert between 6 formats: JSON, YAML, TOML, CSV, XML, TOON
Built-in Analytics: Compare formats instantly - see token savings before you commit
Zero Config: Works out of the box - no setup, no config files needed

Features

Core Capabilities

100% TOON v2.0 Spec Compliant: All 26 specification tests passing
30-60% Token Savings: Verified with benchmarks on real-world data
Multi-Format Support: JSON, YAML, TOML, CSV, XML ↔ TOON
Tabular Optimization: Exceptional efficiency for DataFrame-like structures
Token Analysis: Compare token usage across formats using tiktoken
Type Inference: Automatic type detection and preservation
Strict Validation: Optional strict mode for production safety

Framework Integrations (11)

Pandas: DataFrame ↔ TOON with tabular optimization
Pydantic: BaseModel serialization with validation
LangChain: Document and Message support for RAG systems
FastAPI: Native TOON response class
SQLAlchemy: ORM model serialization and bulk operations
MCP: Model Context Protocol server with 4 tools
LlamaIndex: Node and Document support
Haystack: Document integration for pipelines
DSPy: Example and prediction support
Instructor: Response model integration
Redis: Efficient serialization for RAG metadata and key-value stores

Quick Start

Installation

pip install toonverter

Basic Usage

import toonverter as toon

# Encode Python dict to TOON
data = {"name": "Alice", "age": 30, "city": "NYC"}
toon_str = toon.encode(data)
print(toon_str)
# Output: name: Alice
#         age: 30
#         city: NYC

# Decode TOON back to Python dict
decoded = toon.decode(toon_str)
print(decoded)
# Output: {'name': 'Alice', 'age': 30, 'city': 'NYC'}

# Analyze token usage
report = toon.analyze(data, compare_formats=['json', 'toon'])
print(f"Token savings: {report.max_savings_percentage:.1f}%")
# Output: Token savings: 33.3%

Contents

User Guide

Examples

API Reference

Development