Quick Start

This guide will get you training your first neural network with Entrenar in under 5 minutes.

Your First Neural Network

Let's build a simple linear regression model to learn the function y = 2x + 1.

Step 1: Create a New Project

cargo new entrenar_quickstart
cd entrenar_quickstart

Step 2: Add Dependencies

Edit Cargo.toml:

[dependencies]
entrenar = "0.1"
ndarray = "0.15"

Step 3: Write the Training Code

Edit src/main.rs:

use entrenar::{Tensor, optim::SGD, backward};

fn main() {
    // Training data: y = 2x + 1
    let x_data = vec![1.0, 2.0, 3.0, 4.0];
    let y_data = vec![3.0, 5.0, 7.0, 9.0];

    // Initialize parameters (trainable)
    let mut w = Tensor::from_vec(vec![0.0], true);  // weight
    let mut b = Tensor::from_vec(vec![0.0], true);  // bias

    // Create optimizer
    let mut optimizer = SGD::new(0.01, 0.0);  // learning_rate=0.01, momentum=0.0

    // Training loop
    for epoch in 0..100 {
        let mut total_loss = 0.0;

        for (x, y_true) in x_data.iter().zip(y_data.iter()) {
            // Forward pass: y_pred = w * x + b
            let x_tensor = Tensor::from_vec(vec![*x], false);
            let y_pred = &(&w * &x_tensor) + &b;

            // Compute loss: MSE = (y_pred - y_true)²
            let y_true_tensor = Tensor::from_vec(vec![*y_true], false);
            let diff = &y_pred - &y_true_tensor;
            let loss = &diff * &diff;

            total_loss += loss.data()[0];

            // Backward pass (compute gradients)
            backward(&loss);

            // Update parameters
            optimizer.step(&mut [&mut w, &mut b]);

            // Zero gradients for next iteration
            w.zero_grad();
            b.zero_grad();
        }

        if epoch % 10 == 0 {
            println!("Epoch {}: Loss = {:.6}", epoch, total_loss / x_data.len() as f32);
        }
    }

    // Check learned parameters
    println!("\nLearned parameters:");
    println!("w = {:.4} (expected: 2.0)", w.data()[0]);
    println!("b = {:.4} (expected: 1.0)", b.data()[0]);
}

Step 4: Run the Training

cargo run --release

Expected output:

Epoch 0: Loss = 23.500000
Epoch 10: Loss = 5.123456
Epoch 20: Loss = 1.234567
Epoch 30: Loss = 0.456789
Epoch 40: Loss = 0.123456
Epoch 50: Loss = 0.034567
Epoch 60: Loss = 0.009876
Epoch 70: Loss = 0.002345
Epoch 80: Loss = 0.000567
Epoch 90: Loss = 0.000123

Learned parameters:
w = 1.9987 (expected: 2.0)
b = 1.0024 (expected: 1.0)

Success! Your model learned the linear relationship y = 2x + 1.

Understanding the Code

Let's break down the key components:

1. Tensor Creation

#![allow(unused)]
fn main() {
let mut w = Tensor::from_vec(vec![0.0], true);  // requires_grad=true
}

requires_grad=true: Enables gradient tracking for backpropagation
Parameters must be mutable (mut) to update during training

2. Forward Pass

#![allow(unused)]
fn main() {
let y_pred = &(&w * &x_tensor) + &b;  // y = w * x + b
}

Operators (*, +) are overloaded for tensors
Use references (&) to avoid moving tensors

3. Loss Computation

#![allow(unused)]
fn main() {
let diff = &y_pred - &y_true_tensor;
let loss = &diff * &diff;  // MSE = (y_pred - y_true)²
}

Mean Squared Error (MSE) is a common regression loss
Loss must be a scalar for backpropagation

4. Backward Pass

#![allow(unused)]
fn main() {
backward(&loss);
}

Computes gradients for all tensors with requires_grad=true
Gradients accumulate in tensor.grad()

5. Optimizer Step

#![allow(unused)]
fn main() {
optimizer.step(&mut [&mut w, &mut b]);
}

Updates parameters: w = w - learning_rate * grad_w
SGD, Adam, AdamW all use the same interface

6. Zero Gradients

#![allow(unused)]
fn main() {
w.zero_grad();
b.zero_grad();
}

Critical: Gradients accumulate by default
Always zero gradients after each optimizer step

Next Steps

Try Different Optimizers

Replace SGD with Adam for adaptive learning rates:

#![allow(unused)]
fn main() {
use entrenar::optim::Adam;

let mut optimizer = Adam::default_params(0.01);  // learning_rate=0.01
}

Add More Layers

Build a multi-layer perceptron:

#![allow(unused)]
fn main() {
use entrenar::autograd::ops::{matmul, relu};

// Hidden layer: h = relu(W1 * x + b1)
let h = relu(&(&matmul(&w1, &x, 10, 1, 1) + &b1));

// Output layer: y = W2 * h + b2
let y_pred = &matmul(&w2, &h, 1, 10, 1) + &b2;
}

Use LoRA for Fine-Tuning

Apply LoRA to large pretrained weights:

#![allow(unused)]
fn main() {
use entrenar::lora::LoRALayer;

// Freeze base weights, train only LoRA adapters
let base_weight = Tensor::from_vec(vec![...], false);  // frozen
let lora = LoRALayer::new(base_weight, 256, 256, rank=16, alpha=32.0);

let output = lora.forward(&input);
}

Enable QLoRA for Memory Efficiency

Reduce memory by 75% with 4-bit quantization:

#![allow(unused)]
fn main() {
use entrenar::lora::QLoRALayer;

// Base weights quantized to 4-bit, adapters remain float32
let qlora = QLoRALayer::new(base_weight, 256, 256, rank=16, alpha=32.0);

let output = qlora.forward(&input);  // Dequantizes on-the-fly
}

Common Patterns

Gradient Checking

Validate gradients with finite differences:

#![allow(unused)]
fn main() {
#[cfg(test)]
mod tests {
    use entrenar::autograd::test_utils::check_gradient;

    #[test]
    fn test_my_operation() {
        let x = Tensor::from_vec(vec![1.0, 2.0], true);
        let output = my_operation(&x);

        // Verify gradients are correct (ε=1e-3, threshold=0.2)
        assert!(check_gradient(&output, &x, 1e-3, 0.2));
    }
}
}

Learning Rate Scheduling

Decay learning rate over time:

#![allow(unused)]
fn main() {
use entrenar::optim::schedulers::CosineScheduler;

let scheduler = CosineScheduler::new(
    initial_lr=0.1,
    min_lr=0.001,
    total_steps=1000
);

for step in 0..1000 {
    let lr = scheduler.get_lr(step);
    optimizer.set_lr(lr);

    // ... training step ...
}
}

Gradient Clipping

Prevent exploding gradients:

#![allow(unused)]
fn main() {
use entrenar::optim::clip_grad_norm;

// Clip gradients to max norm of 1.0
clip_grad_norm(&mut [&mut w, &mut b], 1.0);

optimizer.step(&mut [&mut w, &mut b]);
}

Performance Tips

1. Use Release Mode

Always train with optimizations enabled:

cargo run --release  # 10-100x faster than debug builds

2. Enable SIMD

SIMD acceleration activates automatically for tensors ≥16 elements:

#![allow(unused)]
fn main() {
// SIMD-accelerated (fast)
let large_tensor = Tensor::from_vec(vec![0.0; 1024], true);

// Scalar fallback (slower)
let small_tensor = Tensor::from_vec(vec![0.0; 8], true);
}

3. Batch Operations

Process multiple samples together:

#![allow(unused)]
fn main() {
// Batch matrix multiplication
let batch_output = matmul(&weights, &batch_input, d_out, d_in, batch_size);
}

Troubleshooting

Gradients Not Flowing

Problem: Parameters not updating

Solution: Check requires_grad=true and that backward pass is called:

#![allow(unused)]
fn main() {
let mut w = Tensor::from_vec(vec![0.0], true);  // ✅ requires_grad=true
backward(&loss);  // ✅ Must call backward
}

Loss Not Decreasing

Problem: Training is stuck

Solutions:

Check learning rate (try 0.001, 0.01, 0.1)
Verify loss computation is correct
Check gradients aren't being zeroed too early
Try different optimizer (Adam instead of SGD)

Stack Overflow in Tests

Problem: Gradient checking causes stack overflow

Solution: Increase stack size:

RUST_MIN_STACK=8388608 cargo test

What's Next?

First Training Loop - Build a complete training pipeline with validation
Core Concepts - Deep dive into Entrenar's architecture
Examples - More practical examples

Ready for a complete training pipeline? Continue to First Training Loop →

Entrenar - Training & Optimization Library