toonverter Logo

User Guide

  • Getting Started
    • What is TOON?
    • Why Use TOON Converter?
      • 100% Spec Compliance
      • Comprehensive Integrations
      • Production Ready
    • Token Savings Examples
      • Simple Object
      • Tabular Data
    • Use Cases
      • RAG Systems
      • LLM Prompts
      • API Responses
      • Data Pipelines
    • Next Steps
    • Questions or Issues?
  • Installation
    • Basic Installation
    • Individual Framework Integrations
      • Data Science
      • AI/LLM Frameworks
      • Web Frameworks
      • Protocols
    • Grouped Integrations
    • CLI Tools
    • Complete Installation
    • Development Installation
      • Clone the Repository
      • Install in Editable Mode
    • Verify Installation
      • Check Installed Features
    • Requirements
      • Core Dependencies
      • Optional Dependencies
      • Development Dependencies
    • Platform Support
    • Troubleshooting
      • Import Errors
      • Type Checking Issues
      • Version Conflicts
    • Next Steps
  • Quick Start
    • Simple Facade API
      • Encoding and Decoding
      • Working with Files
      • Token Analysis
      • Listing Formats
    • Object-Oriented API
      • Custom Encoder
      • Custom Decoder
      • Stateful Converter
      • Token Analyzer
    • Working with Different Data Types
      • Nested Objects
      • Arrays
      • Type Preservation
    • Command-Line Interface
      • Basic Commands
    • Integration Examples
      • Pandas DataFrame
      • Pydantic Models
      • LangChain Documents
      • FastAPI Responses
    • Best Practices
    • Next Steps
  • TOON Format Specification v2.0
    • Overview
    • Three Root Forms
      • 1. Object Form (Default)
      • 2. Array Form
      • 3. Primitive Form
    • Three Array Forms
      • 1. Inline Array
      • 2. Tabular Array
      • 3. List Array
    • String Quoting Rules
      • Strings That Don’t Need Quotes
    • Number Canonical Form
      • Valid Numbers
      • Normalization Rules
    • Delimiters
      • Comma (Default)
      • Tab
      • Pipe
    • Escape Sequences
    • Indentation
      • Rules
      • Example
    • Type Annotations
    • Complete Example
    • Token Savings
      • Real-World Examples
    • Reference
    • See Also
  • Configuration
    • Encoder Configuration
      • Basic Options
      • Delimiter Options
      • Compact Mode
    • Decoder Configuration
      • Basic Options
      • Strict Mode
    • Converter Configuration
    • Analyzer Configuration
      • Tokenizer Models
    • Global Configuration
    • Environment Variables
    • Integration-Specific Configuration
      • Pandas Integration
      • Pydantic Integration
      • SQLAlchemy Integration
      • LangChain Integration
    • Custom Format Adapters
    • Plugin Development
    • Best Practices
      • Production Deployments
      • Performance Optimization
      • Type Safety
    • See Also
  • Semantic Deduplication
    • Installation
    • CLI Usage
    • Python API
      • Advanced Usage: Custom Text Extraction
    • Performance
  • Schema Tools
    • Schema Inference
      • CLI Usage
      • Python API
    • Schema Validation
      • CLI Usage
      • Python API
    • Schema Merging
      • CLI Usage
      • Python API

Integration Guides

  • Pandas Integration
    • Installation
    • Basic Usage
      • DataFrame to TOON
      • TOON to DataFrame
    • Configuration Options
      • Include Index
      • Date Formatting
    • Token Savings
    • Use Cases
      • Data Storage
      • LLM Context
      • API Responses
    • Type Handling
      • Numeric Types
      • DateTime Types
      • Categorical Types
    • Performance
    • See Also
  • Pydantic Integration
    • Installation
    • Basic Usage
    • See Also
  • LangChain Integration
    • Installation
    • Document Conversion
    • Message Conversion
    • Use Cases
      • Vector Database Storage
      • RAG Pipeline
    • Configuration
    • See Also
  • FastAPI Integration
    • Installation
    • Basic Usage
      • Response Automatically
    • Token Savings
    • Use Cases
    • See Also
  • Sqlalchemy Integration
    • Installation
    • Basic Usage
    • See Also
  • Mcp Integration
    • Installation
    • Basic Usage
    • See Also
  • Llamaindex Integration
    • Installation
    • Basic Usage
    • See Also
  • Haystack Integration
    • Installation
    • Basic Usage
    • See Also
  • Dspy Integration
    • Installation
    • Basic Usage
    • See Also
  • Instructor Integration
    • Installation
    • Basic Usage
    • See Also
  • Redis Integration
    • Quick Start
    • API Reference

Examples

  • Basic Usage
  • Tabular Data
  • Nested Structures
  • Token Analysis
  • CLI Reference
    • Global Options
    • Commands
      • convert
      • encode
      • decode
      • analyze
      • deduplicate
      • infer
      • validate
      • schema-merge
      • diff
      • compress / decompress
      • formats
  • Semantic Deduplication
  • Schema Tools
  • Custom Adapters
  • Smart Dictionary Compression
  • Structural Diff
  • Vision Optimization

API Reference

  • Facade API
    • Core Functions
    • Example Usage
  • Analysis API
    • FormatComparator
      • FormatComparator.__init__()
      • FormatComparator.compare_formats()
    • ReportFormatter
      • ReportFormatter.format_analysis()
      • ReportFormatter.format_comparison()
      • ReportFormatter.format_json()
    • TiktokenCounter
      • TiktokenCounter.MODEL_ENCODINGS
      • TiktokenCounter.__init__()
      • TiktokenCounter.analyze()
      • TiktokenCounter.count_tokens()
      • TiktokenCounter.model_name
    • analyze_text()
    • compare()
    • count_tokens()
    • format_report()
    • Deduplication
  • Converter API
    • Example Usage
  • Core API
    • ComparisonReport
      • ComparisonReport.analyses
      • ComparisonReport.best_format
      • ComparisonReport.worst_format
      • ComparisonReport.recommendations
      • ComparisonReport.__init__()
      • ComparisonReport.max_savings_percentage
      • ComparisonReport.analyses
      • ComparisonReport.best_format
      • ComparisonReport.worst_format
      • ComparisonReport.recommendations
    • ConversionError
    • ConversionResult
      • ConversionResult.success
      • ConversionResult.source_format
      • ConversionResult.target_format
      • ConversionResult.source_tokens
      • ConversionResult.target_tokens
      • ConversionResult.savings_percentage
      • ConversionResult.data
      • ConversionResult.error
      • ConversionResult.metadata
      • ConversionResult.__init__()
      • ConversionResult.__post_init__()
      • ConversionResult.data
      • ConversionResult.error
      • ConversionResult.savings_percentage
      • ConversionResult.source_tokens
      • ConversionResult.target_tokens
      • ConversionResult.success
      • ConversionResult.source_format
      • ConversionResult.target_format
      • ConversionResult.metadata
    • DecodeOptions
      • DecodeOptions.strict
      • DecodeOptions.type_inference
      • DecodeOptions.delimiter
      • DecodeOptions.__init__()
      • DecodeOptions.delimiter
      • DecodeOptions.strict
      • DecodeOptions.type_inference
    • DecodingError
    • DefaultFormatRegistry
      • DefaultFormatRegistry.__init__()
      • DefaultFormatRegistry.__new__()
      • DefaultFormatRegistry.clear()
      • DefaultFormatRegistry.get()
      • DefaultFormatRegistry.is_supported()
      • DefaultFormatRegistry.list_formats()
      • DefaultFormatRegistry.register()
      • DefaultFormatRegistry.unregister()
    • EncodeOptions
      • EncodeOptions.indent
      • EncodeOptions.delimiter
      • EncodeOptions.length_marker
      • EncodeOptions.compact
      • EncodeOptions.sort_keys
      • EncodeOptions.ensure_ascii
      • EncodeOptions.max_line_length
      • EncodeOptions.token_budget
      • EncodeOptions.optimization_policy
      • EncodeOptions.__init__()
      • EncodeOptions.compact
      • EncodeOptions.create_compact()
      • EncodeOptions.delimiter
      • EncodeOptions.ensure_ascii
      • EncodeOptions.indent
      • EncodeOptions.length_marker
      • EncodeOptions.max_line_length
      • EncodeOptions.optimization_policy
      • EncodeOptions.readable()
      • EncodeOptions.sort_keys
      • EncodeOptions.tabular()
      • EncodeOptions.token_budget
    • EncodingError
    • FileOperationError
    • FormatAdapter
      • FormatAdapter.decode()
      • FormatAdapter.decode_stream()
      • FormatAdapter.encode()
      • FormatAdapter.encode_stream()
      • FormatAdapter.format_name
      • FormatAdapter.supports_streaming()
      • FormatAdapter.validate()
    • FormatNotSupportedError
    • FormatRegistry
      • FormatRegistry.get()
      • FormatRegistry.is_supported()
      • FormatRegistry.list_formats()
      • FormatRegistry.register()
      • FormatRegistry.unregister()
    • Plugin
      • Plugin.cleanup()
      • Plugin.initialize()
      • Plugin.name
      • Plugin.register()
      • Plugin.version
    • PluginError
    • TokenAnalysis
      • TokenAnalysis.format
      • TokenAnalysis.token_count
      • TokenAnalysis.model
      • TokenAnalysis.encoding
      • TokenAnalysis.metadata
      • TokenAnalysis.__init__()
      • TokenAnalysis.encoding
      • TokenAnalysis.model
      • TokenAnalysis.format
      • TokenAnalysis.token_count
      • TokenAnalysis.metadata
    • TokenCountError
    • TokenCounter
      • TokenCounter.analyze()
      • TokenCounter.count_tokens()
      • TokenCounter.model_name
    • ToonConverterError
    • ValidationError
    • get_registry()
  • Decoder API
    • Facade Decoder
    • Decoder Implementation
  • Differ API
  • Encoder API
    • Facade Encoder
    • Encoder Implementation
  • Formats API
  • Integrations API
    • Pandas Integration
    • FastAPI Integration
    • LangChain Integration
    • Pydantic Integration
    • SQLAlchemy Integration
    • MCP Integration
    • LlamaIndex Integration
    • Haystack Integration
    • DSPy Integration
    • Instructor Integration
    • Redis Integration
  • Multimodal API
  • Optimization API
    • Context Optimization
      • toonverter.optimization.ContextOptimizer
        • toonverter.optimization.ContextOptimizer.optimize()
    • Smart Compression
      • toonverter.optimization.SmartCompressor
        • toonverter.optimization.SmartCompressor.compress()
        • toonverter.optimization.SmartCompressor.decompress()
  • Plugins API
  • RAG API
  • Schema API
  • Utils API

Development

  • Contributing
    • Quick Start
    • Development Setup
    • Running Tests
    • Code Quality
    • Authors & Contributors
    • Contribution Areas
    • How to Report Bugs
    • How to Request Features
    • Code Review Process
    • License
    • Thank You!
  • Architecture
  • Testing
  • Changelog
    • Latest Releases
      • Version 1.0.2 (2025-01-17)
      • Version 1.0.1 (2025-01-17)
      • Version 1.0.0 (2025-01-17)
toonverter
  • Search

© Copyright 2025, Be-Wagile India.

Built with Sphinx using a theme provided by Read the Docs.