- Introduction
- Getting Started
- 1. Installation
- 2. Quick Start
- 3. First Training Loop
- 4. Core Concepts
- Architecture
- 5. Overview
- 6. Design Philosophy
- 7. Module Organization
- 8. Type System
- 9. Memory Management
- Autograd Engine
- 10. What is Automatic Differentiation?
- 11. Tape-Based Computation Graphs
- 12. Tensor Operations
- 12.1. Matrix Multiplication
- 12.2. Activations (ReLU, GELU, Swish)
- 12.3. Layer Normalization
- 12.4. Attention Mechanism
- 13. Backward Pass
- 14. Gradient Computation
- 15. Finite Difference Validation
- Optimizers
- 16. Overview
- 17. Stochastic Gradient Descent (SGD)
- 18. Adam Optimizer
- 19. AdamW (Decoupled Weight Decay)
- 20. Learning Rate Schedulers
- 20.1. Cosine Annealing
- 20.2. Step Decay
- 20.3. Exponential Decay
- 21. Gradient Clipping
- 22. SIMD-Accelerated Updates
- 23. Optimizer Theory
- LoRA (Low-Rank Adaptation)
- 24. What is LoRA?
- 25. Parameter-Efficient Fine-Tuning
- 26. LoRA Layer Architecture
- 26.1. Low-Rank Matrices A and B
- 26.2. Scaling Factor (alpha/rank)
- 26.3. Merge and Unmerge
- 27. Target Module Selection
- 28. Gradient Flow Isolation
- 29. Adapter Persistence
- 29.1. Saving Adapters
- 29.2. Loading Adapters
- 29.3. Sharing Adapters
- QLoRA (Quantized LoRA)
- 30. Memory-Efficient Fine-Tuning
- 31. 4-bit Quantization
- 31.1. Block-Wise Quantization
- 31.2. Scale Factors
- 31.3. Quantization/Dequantization
- 32. QLoRA Layer
- 33. On-the-Fly Dequantization
- 34. Memory Benchmarks
- 34.1. LoRA vs QLoRA Comparison
- 34.2. Transformer Model Benchmarks
- 34.3. Compression Ratios
- 35. Trade-offs and Best Practices
- Model Merging
- 36. Overview
- 37. TIES Algorithm
- 38. DARE Algorithm
- 39. SLERP Algorithm
- 40. Multi-Model Ensembles
- 41. Merge Best Practices
- Knowledge Distillation
- 42. What is Distillation?
- 43. Temperature-Scaled KL Divergence
- 44. Multi-Teacher Ensemble
- 45. Progressive Layer-Wise
- 46. Distillation Loss Functions
- 47. Student-Teacher Architecture
- Training Loops
- 48. Trainer API
- 49. Callback System
- 50. Train Config
- 51. Basic Training Loop
- 52. Batching and Data Loading
- 53. Loss Functions
- 54. Validation and Testing
- 55. Checkpointing
- 56. Early Stopping
- 57. Curriculum Learning
- 58. Explainability
- Real-Time Monitoring
- 59. Overview
- 60. Experiment Tracking
- 61. Quality Gates (Jidoka)
- 62. Metrics Collection
- 63. Terminal Dashboard
- 64. Drift Detection
- 65. Andon Alerting (Jidoka)
- 66. Model Lineage
- 67. Export Formats
- 68. Hansei Reports
- Dashboard
- 69. Overview
- 70. DashboardSource Trait
- 71. WASM Bindings
- Ecosystem Integration
- 72. Overview
- 73. Batuta Integration
- 74. Realizar GGUF Export
- 75. Ruchy Session Bridge
- Model I/O
- 76. Overview
- 77. Save Models
- 78. Load Models
- 79. Model Metadata
- 80. Supported Formats
- 80.1. SafeTensors Format
- 80.2. JSON Format
- 80.3. YAML Format
- 80.4. GGUF Format
- Command-Line Interface
- 81. CLI Overview
- 82. Research Commands
- 83. Benchmark Commands
- Declarative Training
- 84. Overview
- 85. YAML Mode Training (v1.0)
- 86. YAML Examples Catalog
- 87. Toyota Way QA Process
- 88. YAML Configuration
- 89. train_from_yaml Function
- 90. Configuration Schema
- 91. Optimizer Builders
- 92. Model Builders
- API Reference
- 93. Tensor API
- 94. Autograd Operations
- 95. Optimizer API
- 96. LoRA API
- 97. QLoRA API
- 98. Configuration System
- 99. Error Handling
- Examples
- 100. Linear Regression with Autograd
- 101. Training a Simple MLP
- 102. Fine-Tuning with LoRA
- 103. Memory-Efficient QLoRA
- 104. Custom Loss Functions
- 105. Learning Rate Scheduling
- 106. Gradient Clipping
- 107. Adapter Sharing
- Development Guide
- 108. Contributing
- 109. EXTREME TDD Methodology
- 110. Testing Strategy
- 110.1. Unit Tests
- 110.2. Property-Based Tests
- 110.3. Gradient Checking Tests
- 110.4. Mutation Testing
- 111. Quality Gates
- 111.1. Pre-Commit Hooks
- 111.2. Continuous Integration
- 111.3. Code Coverage
- 111.4. Clippy Linting
- 112. Benchmarking
- 113. PMAT Toyota Workflow
- Best Practices
- 114. Optimizer Selection
- 115. Learning Rate Tuning
- 116. LoRA Configuration
- 117. Memory Optimization
- 118. Gradient Stability
- 119. Debugging Training Issues
- 120. Performance Profiling
- Advanced Topics
- 121. Custom Backward Passes
- 122. Implementing New Optimizers
- 123. Custom LoRA Variants
- 124. Advanced Quantization
- 125. Distributed Training
- 126. Model Parallelism
- 127. Compiler-in-the-Loop (CITL)
- Sovereign Deployment
- 128. Overview
- Academic Research
- 129. Overview
- Specifications
- 130. Autograd Specification
- 131. Optimizer Specification
- 132. LoRA Specification
- 133. Quantization Specification
- 134. Academic Foundations
- Appendix
- 135. Glossary
- 136. Mathematical Notation
- 137. References
- 138. FAQ
- 139. Changelog
- 140. Migration Guide
- 141. Benchmarking Results