Discriminator Architecture - Entrenar - Training & Optimization Library
Description: A comprehensive guide to building neural network training systems with autograd, optimizers, LoRA/QLoRA, and quantization using EXTREME TDD methodology

# Discriminator Architecture

The discriminator classifies code as real or generated.

## Architecture

    Input: tokens ∈ N^seq_len
        ↓
    Embedding(vocab_size → embed_dim)
        ↓
    Flatten to (seq_len * embed_dim)
        ↓
    Linear → ReLU
        ↓
    Linear → ReLU
        ↓
    ...
        ↓
    Linear → 1
        ↓
    Sigmoid → P(real)

## Configuration

    #![allow(unused)]
    fn main() {
    let config = DiscriminatorConfig {
        vocab_size: 100,        // Number of token types
        embed_dim: 64,          // Embedding dimension
        hidden_dims: vec![128, 64],  // MLP layers
        dropout: 0.1,           // Regularization
    };
    }

## Training Objective

Binary cross-entropy loss:

    #![allow(unused)]
    fn main() {
    fn discriminator_loss(
        d_real: f32,    // D(x) for real sample
        d_fake: f32,    // D(G(z)) for generated sample
    ) -> f32 {
        // Maximize: log(D(x)) + log(1 - D(G(z)))
        let real_loss = -d_real.ln();
        let fake_loss = -(1.0 - d_fake).ln();
        real_loss + fake_loss
    }
    }

## Label Smoothing

Improve training stability:

    #![allow(unused)]
    fn main() {
    fn train_with_smoothing(&mut self, real: &[u32], fake: &[u32]) {
        // Soft labels: 0.9 for real, 0.1 for fake
        let real_label = 0.9;
        let fake_label = 0.1;

        let d_real = self.discriminator.forward(real);
        let d_fake = self.discriminator.forward(fake);

        let loss = bce(d_real, real_label) + bce(d_fake, fake_label);
        self.backward(loss);
    }
    }

## Feature Matching

Alternative to standard GAN loss:

    #![allow(unused)]
    fn main() {
    fn feature_matching_loss(
        discriminator: &Discriminator,
        real: &[u32],
        fake: &[u32],
    ) -> f32 {
        let real_features = discriminator.intermediate_features(real);
        let fake_features = discriminator.intermediate_features(fake);

        mse(&real_features, &fake_features)
    }
    }