1. Introduction
  3. Getting Started
  4. Installation
  5. Quick Start
  6. First Training Loop
  7. Core Concepts
  9. Architecture
  10. Overview
  11. Design Philosophy
  12. Module Organization
  13. Type System
  14. Memory Management
  16. Autograd Engine
  17. What is Automatic Differentiation?
  18. Tape-Based Computation Graphs
  19. Tensor Operations
    1. Matrix Multiplication
    2. Activations (ReLU, GELU, Swish)
    3. Layer Normalization
    4. Attention Mechanism
  21. Backward Pass
  22. Gradient Computation
  23. Finite Difference Validation
  25. Optimizers
  26. Overview
  27. Stochastic Gradient Descent (SGD)
  28. Adam Optimizer
  29. AdamW (Decoupled Weight Decay)
  30. Learning Rate Schedulers
    1. Cosine Annealing
    2. Step Decay
    3. Exponential Decay
  32. Gradient Clipping
  33. SIMD-Accelerated Updates
  34. Optimizer Theory
  36. LoRA (Low-Rank Adaptation)
  37. What is LoRA?
  38. Parameter-Efficient Fine-Tuning
  39. LoRA Layer Architecture
    1. Low-Rank Matrices A and B
    2. Scaling Factor (alpha/rank)
    3. Merge and Unmerge
  41. Target Module Selection
  42. Gradient Flow Isolation
  43. Adapter Persistence
    1. Saving Adapters
    2. Loading Adapters
    3. Sharing Adapters
  46. QLoRA (Quantized LoRA)
  47. Memory-Efficient Fine-Tuning
  48. 4-bit Quantization
    1. Block-Wise Quantization
    2. Scale Factors
    3. Quantization/Dequantization
  50. QLoRA Layer
  51. On-the-Fly Dequantization
  52. Memory Benchmarks
    1. LoRA vs QLoRA Comparison
    2. Transformer Model Benchmarks
    3. Compression Ratios
  54. Trade-offs and Best Practices
  56. Model Merging
  57. Overview
  58. TIES Algorithm
  59. DARE Algorithm
  60. SLERP Algorithm
  61. Multi-Model Ensembles
  62. Merge Best Practices
  64. Knowledge Distillation
  65. What is Distillation?
  66. Temperature-Scaled KL Divergence
  67. Multi-Teacher Ensemble
  68. Progressive Layer-Wise
  69. Distillation Loss Functions
  70. Student-Teacher Architecture
  72. Training Loops
  73. Trainer API
  74. Train Config
  75. Basic Training Loop
  76. Batching and Data Loading
  77. Loss Functions
  78. Validation and Testing
  79. Checkpointing
  80. Early Stopping
  82. Model I/O
  83. Overview
  84. Save Models
  85. Load Models
  86. Model Metadata
  87. Supported Formats
    1. JSON Format
    2. YAML Format
    3. GGUF Format
  90. Declarative Training
  91. Overview
  92. YAML Configuration
  93. train_from_yaml Function
  94. Configuration Schema
  95. Optimizer Builders
  96. Model Builders
  98. API Reference
  99. Tensor API
  100. Autograd Operations
  101. Optimizer API
  102. LoRA API
  103. QLoRA API
  104. Configuration System
  105. Error Handling
  107. Examples
  108. Linear Regression with Autograd
  109. Training a Simple MLP
  110. Fine-Tuning with LoRA
  111. Memory-Efficient QLoRA
  112. Custom Loss Functions
  113. Learning Rate Scheduling
  114. Gradient Clipping
  115. Adapter Sharing
  117. Development Guide
  118. Contributing
  119. EXTREME TDD Methodology
  120. Testing Strategy
    1. Unit Tests
    2. Property-Based Tests
    3. Gradient Checking Tests
    4. Mutation Testing
  122. Quality Gates
    1. Pre-Commit Hooks
    2. Continuous Integration
    3. Code Coverage
    4. Clippy Linting
  124. Benchmarking
  125. PMAT Toyota Workflow
  127. Best Practices
  128. Optimizer Selection
  129. Learning Rate Tuning
  130. LoRA Configuration
  131. Memory Optimization
  132. Gradient Stability
  133. Debugging Training Issues
  134. Performance Profiling
  136. Advanced Topics
  137. Custom Backward Passes
  138. Implementing New Optimizers
  139. Custom LoRA Variants
  140. Advanced Quantization
  141. Distributed Training
  142. Model Parallelism
  144. Specifications
  145. Autograd Specification
  146. Optimizer Specification
  147. LoRA Specification
  148. Quantization Specification
  149. Academic Foundations
  151. Appendix
  152. Glossary
  153. Mathematical Notation
  154. References
  155. FAQ
  156. Changelog
  157. Migration Guide
  158. Benchmarking Results