Generator Architecture - Entrenar - Training & Optimization Library
Description: A comprehensive guide to building neural network training systems with autograd, optimizers, LoRA/QLoRA, and quantization using EXTREME TDD methodology

# Generator Architecture

The generator maps latent vectors to code token sequences.

## Architecture

    Input: z ∈ R^latent_dim
        ↓
    Linear(latent_dim → hidden[0])
        ↓
    ReLU + BatchNorm (optional)
        ↓
    Linear(hidden[0] → hidden[1])
        ↓
    ReLU + BatchNorm (optional)
        ↓
    ...
        ↓
    Linear(hidden[-1] → vocab_size * max_seq_len)
        ↓
    Reshape to (max_seq_len, vocab_size)
        ↓
    Softmax per position
        ↓
    Argmax → Token IDs

## Configuration

    #![allow(unused)]
    fn main() {
    let config = GeneratorConfig {
        latent_dim: 128,        // Size of noise vector
        hidden_dims: vec![256, 512, 256],  // MLP layers
        vocab_size: 100,        // Number of token types
        max_seq_len: 64,        // Maximum output length
        dropout: 0.1,           // Regularization
        batch_norm: true,       // Stabilize training
    };
    }

## Initialization

Xavier initialization for stable gradients:

    #![allow(unused)]
    fn main() {
    fn xavier_init(fan_in: usize, fan_out: usize) -> f32 {
        let limit = (6.0 / (fan_in + fan_out) as f32).sqrt();
        rand::random::<f32>() * 2.0 * limit - limit
    }
    }

## Generation

    #![allow(unused)]
    fn main() {
    let generator = Generator::new(config);
    let z = LatentCode::sample(&mut rng, 128);
    let tokens: Vec<u32> = generator.generate(&z);
    }

## Temperature Sampling

Control diversity vs quality:

    #![allow(unused)]
    fn main() {
    fn generate_with_temperature(&self, z: &LatentCode, temp: f32) -> Vec<u32> {
        let logits = self.forward(z);

        logits.chunks(self.vocab_size)
            .map(|chunk| {
                let scaled: Vec<f32> = chunk.iter()
                    .map(|x| x / temp)
                    .collect();
                softmax_sample(&scaled)
            })
            .collect()
    }
    }