# Processor Base Struct Refactor

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Eliminate per-processor boilerplate by introducing a `ProcessorBase` struct that provides default implementations for `config_json()`, `max_jobs()`, `clean()`, `processor_type()`, `description()`, and `auto_detect()`.

**Architecture:** Add a `ProcessorBase` struct holding common fields (config as `serde_json::Value`, processor name, description, processor type, scan config ref). Each processor embeds `ProcessorBase` and delegates boilerplate trait methods to it. A `delegate_base!` macro generates the delegation one-liners. Processors only implement `execute()`, `discover()`, and `required_tools()` manually — everything else comes from the base.

**Tech Stack:** Rust, serde_json, existing `ProductDiscovery` trait

---

## File Structure

| File | Action | Responsibility |
|------|--------|---------------|
| `src/processors/base.rs` | Create | `ProcessorBase` struct + methods for all boilerplate |
| `src/processors/mod.rs` | Modify | Add `mod base`, `pub use base::ProcessorBase`, add `delegate_base!` macro |
| `src/processors/generators/mod.rs` | Modify | Replace `impl_generator!` macro with `delegate_base!` |
| `src/processors/generators/marp.rs` | Modify | First migration: use `ProcessorBase` |
| `src/processors/generators/mako.rs` | Modify | Migration |
| `src/processors/generators/jinja2.rs` | Modify | Migration |
| `src/processors/generators/tera.rs` | Modify | Migration |
| `src/processors/generators/pdflatex.rs` | Modify | Migration |
| `src/processors/generators/pdfunite.rs` | Modify | Migration |
| `src/processors/generators/rust_single_file.rs` | Modify | Migration |
| `src/processors/generators/generator.rs` | Modify | Migration |
| `src/processors/generators/tags.rs` | Modify | Migration |
| `src/processors/generators/cc_single_file/mod.rs` | Modify | Migration |
| `src/processors/generators/linux_module.rs` | Modify | Migration |
| `src/processors/generators/explicit.rs` | Modify | Migration (partial - no standard config) |
| `src/processors/generators/pandoc.rs` | Modify | Migration from `impl_generator!` |
| `src/processors/generators/sass.rs` | Modify | Migration from `impl_generator!` |
| `src/processors/generators/chromium.rs` | Modify | Migration from `impl_generator!` |
| (remaining impl_generator! users) | Modify | Migration from `impl_generator!` |
| `src/processors/checkers/mod.rs` | Modify | Update `impl_checker!` to use `ProcessorBase` |
| `src/processors/checkers/zspell.rs` | Modify | Migration of manual checker |
| `src/processors/checkers/clippy.rs` | Modify | Migration of manual checker |
| `src/processors/checkers/make.rs` | Modify | Migration of manual checker |
| `src/processors/checkers/aspell.rs` | Modify | Migration of manual checker |
| `src/processors/checkers/mdl.rs` | Modify | Migration of manual checker |
| `src/processors/checkers/markdownlint.rs` | Modify | Migration of manual checker |
| `tests/tests_mod/build.rs` | Verify | Run existing tests to confirm no regressions |

---

### Task 1: Create ProcessorBase struct

**Files:**
- Create: `src/processors/base.rs`
- Modify: `src/processors/mod.rs`

- [ ] **Step 1: Create `src/processors/base.rs` with the base struct**

```rust
use serde::Serialize;
use crate::config::ScanConfig;
use crate::graph::Product;
use crate::processors::ProcessorType;

/// Common base for all processors. Holds fields needed by boilerplate
/// ProductDiscovery methods so each processor doesn't repeat them.
pub struct ProcessorBase {
    /// Processor name constant (e.g., "marp", "pylint")
    pub name: &'static str,
    /// Human-readable description
    pub description: &'static str,
    /// Generator or Checker
    pub processor_type: ProcessorType,
}

impl ProcessorBase {
    pub fn generator(name: &'static str, description: &'static str) -> Self {
        Self { name, description, processor_type: ProcessorType::Generator }
    }

    pub fn checker(name: &'static str, description: &'static str) -> Self {
        Self { name, description, processor_type: ProcessorType::Checker }
    }

    pub fn description(&self) -> &str {
        self.description
    }

    pub fn processor_type(&self) -> ProcessorType {
        self.processor_type
    }

    pub fn config_json<C: Serialize>(config: &C) -> Option<String> {
        serde_json::to_string(config).ok()
    }

    pub fn clean(product: &Product, name: &str, verbose: bool) -> anyhow::Result<usize> {
        crate::processors::clean_outputs(product, name, verbose)
    }

    pub fn auto_detect(scan: &ScanConfig, file_index: &crate::file_index::FileIndex) -> bool {
        crate::processors::scan_root_valid(scan) && !file_index.scan(scan, true).is_empty()
    }
}
```

- [ ] **Step 2: Add `mod base` and `pub use` to `src/processors/mod.rs`**

Add near the top of `src/processors/mod.rs`, after the other `mod` declarations:

```rust
mod base;
pub use base::ProcessorBase;
```

- [ ] **Step 3: Add `delegate_base!` macro to `src/processors/mod.rs`**

Add after the `pub use base::ProcessorBase;` line:

```rust
/// Macro to delegate boilerplate ProductDiscovery methods to ProcessorBase.
/// Usage inside `impl ProductDiscovery for MyProcessor { delegate_base!(); ... }`
///
/// Delegates: description, processor_type, config_json, max_jobs, clean (generators only).
/// The processor must have:
///   - `self.base: ProcessorBase`
///   - `self.config` implementing `Serialize` and having `max_jobs: Option<usize>`
///   - `self.config.scan: ScanConfig`
#[macro_export]
macro_rules! delegate_base {
    // Generator variant: includes clean()
    (generator) => {
        fn description(&self) -> &str {
            self.base.description()
        }

        fn processor_type(&self) -> $crate::processors::ProcessorType {
            self.base.processor_type()
        }

        fn auto_detect(&self, file_index: &$crate::file_index::FileIndex) -> bool {
            $crate::processors::ProcessorBase::auto_detect(&self.config.scan, file_index)
        }

        fn config_json(&self) -> Option<String> {
            $crate::processors::ProcessorBase::config_json(&self.config)
        }

        fn max_jobs(&self) -> Option<usize> {
            self.config.max_jobs
        }

        fn clean(&self, product: &$crate::graph::Product, verbose: bool) -> anyhow::Result<usize> {
            $crate::processors::ProcessorBase::clean(product, self.base.name, verbose)
        }
    };

    // Checker variant: no clean() (default returns Ok(0))
    (checker) => {
        fn description(&self) -> &str {
            self.base.description()
        }

        fn processor_type(&self) -> $crate::processors::ProcessorType {
            self.base.processor_type()
        }

        fn auto_detect(&self, file_index: &$crate::file_index::FileIndex) -> bool {
            $crate::processors::ProcessorBase::auto_detect(&self.config.scan, file_index)
        }

        fn config_json(&self) -> Option<String> {
            $crate::processors::ProcessorBase::config_json(&self.config)
        }

        fn max_jobs(&self) -> Option<usize> {
            self.config.max_jobs
        }
    };
}
```

- [ ] **Step 4: Build and verify compilation**

Run: `cargo build`
Expected: Compiles with no errors (macro and struct are defined but not yet used)

- [ ] **Step 5: Commit**

```bash
git add src/processors/base.rs src/processors/mod.rs
git commit -m "feat: add ProcessorBase struct and delegate_base! macro"
```

---

### Task 2: Migrate MarpProcessor (first manual generator)

**Files:**
- Modify: `src/processors/generators/marp.rs`

- [ ] **Step 1: Run existing tests to establish baseline**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 2: Rewrite MarpProcessor to use ProcessorBase**

Replace the full contents of `src/processors/generators/marp.rs` with:

```rust
use anyhow::{Context, Result};
use std::fs;
use std::process::Command;

use crate::config::MarpConfig;
use crate::file_index::FileIndex;
use crate::graph::{BuildGraph, Product};
use crate::processors::{ProcessorBase, ProductDiscovery, run_command, check_command_output};

use super::DiscoverParams;

fn cleanup_marp_tmp_dirs() {
    let Ok(entries) = fs::read_dir("/tmp") else { return };
    for entry in entries.filter_map(|e| e.ok()) {
        if entry.file_name().to_string_lossy().starts_with("marp-cli-") {
            let _ = fs::remove_dir_all(entry.path());
        }
    }
}

pub struct MarpProcessor {
    base: ProcessorBase,
    config: MarpConfig,
}

impl MarpProcessor {
    pub fn new(config: MarpConfig) -> Self {
        Self {
            base: ProcessorBase::generator(
                crate::processors::names::MARP,
                "Convert Marp slides to PDF/PPTX/HTML",
            ),
            config,
        }
    }
}

impl ProductDiscovery for MarpProcessor {
    delegate_base!(generator);

    fn required_tools(&self) -> Vec<String> {
        vec![self.config.marp_bin.clone(), "node".to_string()]
    }

    fn discover(&self, graph: &mut BuildGraph, file_index: &FileIndex) -> Result<()> {
        let params = DiscoverParams {
            scan: &self.config.scan,
            dep_inputs: &self.config.dep_inputs,
            config: &self.config,
            output_dir: &self.config.output_dir,
            processor_name: crate::processors::names::MARP,
        };
        super::discover_multi_format(graph, file_index, &params, &self.config.formats)
    }

    fn execute(&self, product: &Product) -> Result<()> {
        let input = product.primary_input();
        let output = product.primary_output();

        let format = output.extension()
            .context("marp output has no extension")?
            .to_string_lossy();

        crate::processors::ensure_output_dir(output)?;

        let mut cmd = Command::new(&self.config.marp_bin);
        if format != "html" {
            cmd.arg(format!("--{}", format));
        }
        cmd.arg("--output").arg(output);
        for arg in &self.config.args {
            cmd.arg(arg);
        }
        cmd.arg(input);

        let out = run_command(&mut cmd)?;
        let result = check_command_output(&out, format_args!("marp {}", input.display()));

        cleanup_marp_tmp_dirs();

        result
    }
}
```

- [ ] **Step 3: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 4: Commit**

```bash
git add src/processors/generators/marp.rs
git commit -m "refactor: migrate MarpProcessor to ProcessorBase"
```

---

### Task 3: Migrate template processors (mako, jinja2, tera)

**Files:**
- Modify: `src/processors/generators/mako.rs`
- Modify: `src/processors/generators/jinja2.rs`
- Modify: `src/processors/generators/tera.rs`

These three share an identical `discover()` pattern using `find_templates()`. Each should use `delegate_base!(generator)` but override `auto_detect()` since they use `find_templates` instead of `scan_root_valid`.

- [ ] **Step 1: Rewrite MakoProcessor**

Replace `src/processors/generators/mako.rs`:

```rust
use anyhow::Result;
use std::process::Command;

use crate::config::{MakoConfig, output_config_hash, resolve_extra_inputs};
use crate::file_index::FileIndex;
use crate::graph::{BuildGraph, Product};
use crate::processors::{ProcessorBase, ProductDiscovery, clean_outputs, run_command, check_command_output};

use super::TemplateItem;

fn render_mako(item: &TemplateItem) -> Result<()> {
    crate::processors::ensure_output_dir(&item.output_path)?;

    let source = item.source_path.display().to_string()
        .replace('\\', "\\\\").replace('\'', "\\'");
    let target = item.output_path.display().to_string()
        .replace('\\', "\\\\").replace('\'', "\\'");

    let python_script = format!(
        r#"
import mako.template, mako.lookup
lookup = mako.lookup.TemplateLookup(directories=['.'])
t = mako.template.Template(filename='{}', lookup=lookup)
output = t.render()
with open('{}', 'w') as f:
    f.write(output)
"#,
        source, target
    );

    let mut cmd = Command::new("python3");
    cmd.arg("-c").arg(&python_script);
    let output = run_command(&mut cmd)?;
    check_command_output(&output, format!("mako render {}", item.source_path.display()))
}

pub struct MakoProcessor {
    base: ProcessorBase,
    config: MakoConfig,
}

impl MakoProcessor {
    pub fn new(config: MakoConfig) -> Self {
        Self {
            base: ProcessorBase::generator(
                crate::processors::names::MAKO,
                "Render Mako templates into output files",
            ),
            config,
        }
    }
}

impl ProductDiscovery for MakoProcessor {
    delegate_base!(generator);

    fn auto_detect(&self, file_index: &FileIndex) -> bool {
        super::find_templates(&self.config.scan, file_index).is_ok_and(|t| !t.is_empty())
    }

    fn required_tools(&self) -> Vec<String> {
        vec!["python3".to_string()]
    }

    fn discover(&self, graph: &mut BuildGraph, file_index: &FileIndex) -> Result<()> {
        let items = super::find_templates(&self.config.scan, file_index)?;
        let extra = resolve_extra_inputs(&self.config.dep_inputs)?;

        for item in items {
            let mut inputs = Vec::with_capacity(1 + extra.len());
            inputs.push(item.source_path.clone());
            inputs.extend_from_slice(&extra);
            graph.add_product(
                inputs,
                vec![item.output_path.clone()],
                crate::processors::names::MAKO,
                Some(output_config_hash(&self.config, &[])),
            )?;
        }

        Ok(())
    }

    fn execute(&self, product: &Product) -> Result<()> {
        let item = TemplateItem::new(
            product.primary_input().to_path_buf(),
            product.primary_output().to_path_buf(),
        );
        render_mako(&item)
    }
}
```

- [ ] **Step 2: Rewrite Jinja2Processor (same pattern as Mako)**

Apply the same pattern to `src/processors/generators/jinja2.rs`. Read the file first, keep its `render_jinja2()` function and `execute()` logic, but replace the `impl ProductDiscovery` block with `delegate_base!(generator)` + overrides for `auto_detect`, `required_tools`, `discover`, `execute`.

- [ ] **Step 3: Rewrite TeraProcessor (same pattern, different execute)**

Apply the same pattern to `src/processors/generators/tera.rs`. Read the file first. Tera has more complex `execute()` and `discover()` logic — keep those, but use `delegate_base!(generator)` for the boilerplate and override `auto_detect`.

- [ ] **Step 4: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 5: Commit**

```bash
git add src/processors/generators/mako.rs src/processors/generators/jinja2.rs src/processors/generators/tera.rs
git commit -m "refactor: migrate template processors to ProcessorBase"
```

---

### Task 4: Migrate remaining manual generators

**Files:**
- Modify: `src/processors/generators/pdflatex.rs`
- Modify: `src/processors/generators/pdfunite.rs`
- Modify: `src/processors/generators/rust_single_file.rs`
- Modify: `src/processors/generators/generator.rs`
- Modify: `src/processors/generators/tags.rs`
- Modify: `src/processors/generators/cc_single_file/mod.rs`
- Modify: `src/processors/generators/linux_module.rs`
- Modify: `src/processors/generators/explicit.rs`

For each processor:
1. Add `base: ProcessorBase` field to the struct
2. Initialize it in `new()` with `ProcessorBase::generator(name, description)`
3. Replace boilerplate methods with `delegate_base!(generator)`
4. Keep processor-specific overrides for `execute()`, `discover()`, `required_tools()`, and any custom methods

- [ ] **Step 1: Migrate each processor**

Read each file, apply the pattern. Keep all processor-specific logic. Only replace the boilerplate 6 methods: `description`, `processor_type`, `auto_detect`, `config_json`, `max_jobs`, `clean`.

Note: `explicit.rs` has no `max_jobs` field on its config — use the default trait method (don't include `max_jobs` in delegate, or override to return `None`). `cc_single_file` has custom `discover_for_clean` — keep that override.

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Commit**

```bash
git add src/processors/generators/
git commit -m "refactor: migrate remaining manual generators to ProcessorBase"
```

---

### Task 5: Migrate impl_generator! macro users

**Files:**
- Modify: `src/processors/generators/pandoc.rs`
- Modify: `src/processors/generators/sass.rs`
- Modify: `src/processors/generators/chromium.rs`
- Modify: `src/processors/generators/a2x.rs` (if using macro)
- Modify: `src/processors/generators/drawio.rs`
- Modify: `src/processors/generators/libreoffice.rs`
- Modify: `src/processors/generators/markdown.rs`
- Modify: `src/processors/generators/mermaid.rs`
- Modify: `src/processors/generators/objdump.rs`
- Modify: `src/processors/generators/protobuf.rs`

For each: replace `impl_generator!(...)` invocation with explicit struct + `delegate_base!(generator)` + the discover call (multi_format or single_format). These processors already have an `execute_product()` method — rename to `execute()` inside the trait impl.

- [ ] **Step 1: Migrate each `impl_generator!` user**

Example transformation for pandoc.rs:

Before:
```rust
impl_generator!(PandocProcessor, crate::config::PandocConfig,
    description: "Convert documents using pandoc",
    name: crate::processors::names::PANDOC,
    discover: multi_format, formats_field: formats,
    tool_field: pandoc
);

impl PandocProcessor {
    fn execute_product(&self, product: &Product) -> Result<()> { ... }
}
```

After:
```rust
pub struct PandocProcessor {
    base: ProcessorBase,
    config: crate::config::PandocConfig,
}

impl PandocProcessor {
    pub fn new(config: crate::config::PandocConfig) -> Self {
        Self {
            base: ProcessorBase::generator(
                crate::processors::names::PANDOC,
                "Convert documents using pandoc",
            ),
            config,
        }
    }
}

impl ProductDiscovery for PandocProcessor {
    delegate_base!(generator);

    fn required_tools(&self) -> Vec<String> {
        vec![self.config.pandoc.clone()]
    }

    fn discover(&self, graph: &mut BuildGraph, file_index: &FileIndex) -> Result<()> {
        let params = super::DiscoverParams {
            scan: &self.config.scan,
            dep_inputs: &self.config.dep_inputs,
            config: &self.config,
            output_dir: &self.config.output_dir,
            processor_name: crate::processors::names::PANDOC,
        };
        super::discover_multi_format(graph, file_index, &params, &self.config.formats)
    }

    fn execute(&self, product: &Product) -> Result<()> {
        // ... existing execute_product body ...
    }
}
```

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Remove the `impl_generator!` macro from `src/processors/generators/mod.rs`**

Delete the entire `macro_rules! impl_generator { ... }` block (lines 16-151).

- [ ] **Step 4: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 5: Commit**

```bash
git add src/processors/generators/
git commit -m "refactor: migrate impl_generator! users to ProcessorBase, remove macro"
```

---

### Task 6: Migrate impl_checker! macro to use ProcessorBase

**Files:**
- Modify: `src/processors/checkers/mod.rs`

- [ ] **Step 1: Update the `@build` variant of `impl_checker!`**

In the `@build` arm of `impl_checker!`, replace the individual boilerplate method generations with `delegate_base!(checker)` and add a `base` field to the generated struct.

The macro currently generates: `description`, `auto_detect`, `required_tools`, `discover`, `execute`, `config_json`, `batch`, `max_jobs`.

After refactor, the macro should:
1. Generate struct with `base: ProcessorBase` + `config` fields
2. Generate `new()` that initializes base
3. Use `delegate_base!(checker)` for `description`, `processor_type`, `config_json`, `max_jobs`, `auto_detect`
4. Keep macro-generated `required_tools`, `discover`, `execute`, `batch` (these have variants)

Note: `auto_detect` has 3 variants in the checker macro (scan_root, guard method, no guard). The `delegate_base!` version uses `scan_root_valid`. For guard-based variants, the macro must still override `auto_detect`.

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Commit**

```bash
git add src/processors/checkers/mod.rs
git commit -m "refactor: update impl_checker! to use ProcessorBase"
```

---

### Task 7: Migrate manual checkers to ProcessorBase

**Files:**
- Modify: `src/processors/checkers/zspell.rs`
- Modify: `src/processors/checkers/clippy.rs`
- Modify: `src/processors/checkers/make.rs`
- Modify: `src/processors/checkers/aspell.rs`
- Modify: `src/processors/checkers/mdl.rs`
- Modify: `src/processors/checkers/markdownlint.rs`

For each: add `base: ProcessorBase` field, use `delegate_base!(checker)`, keep processor-specific methods.

- [ ] **Step 1: Migrate each manual checker**

Read each file, apply the pattern. Keep all custom `discover()`, `execute()`, `supports_batch()`, `execute_batch()` logic. Only replace: `description`, `processor_type`, `config_json`, `max_jobs`.

Note: some checkers override `auto_detect` with custom guards — keep those overrides.

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Commit**

```bash
git add src/processors/checkers/
git commit -m "refactor: migrate manual checkers to ProcessorBase"
```

---

### Task 8: Migrate mass generators

**Files:**
- Modify: `src/processors/mass_generators/cargo.rs`
- Modify: `src/processors/mass_generators/cc.rs`
- Modify: `src/processors/mass_generators/gem.rs`
- Modify: `src/processors/mass_generators/jekyll.rs`
- Modify: `src/processors/mass_generators/mdbook.rs`
- Modify: `src/processors/mass_generators/npm.rs`
- Modify: `src/processors/mass_generators/pip.rs`
- Modify: `src/processors/mass_generators/sphinx.rs`

Same pattern: add `base: ProcessorBase`, use `delegate_base!(generator)`, keep custom logic.

- [ ] **Step 1: Migrate each mass generator**

Read each file, apply the pattern. Mass generators typically have custom `discover()` and `execute()` — keep those.

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Commit**

```bash
git add src/processors/mass_generators/
git commit -m "refactor: migrate mass generators to ProcessorBase"
```

---

### Task 9: Migrate LuaProcessor

**Files:**
- Modify: `src/processors/lua_processor.rs`

LuaProcessor is special — it uses dynamic `toml::Value` config, not a typed struct. It may need a custom approach or skip `delegate_base!`.

- [ ] **Step 1: Read the file and assess**

If LuaProcessor can use ProcessorBase (it has name, description, processor_type), add `base` field for those. If `config_json` and `max_jobs` don't apply (no typed config), leave those as-is.

- [ ] **Step 2: Build and run tests**

Run: `cargo test 2>&1 | tail -5`
Expected: All 340 tests pass

- [ ] **Step 3: Commit**

```bash
git add src/processors/lua_processor.rs
git commit -m "refactor: migrate LuaProcessor to ProcessorBase where applicable"
```

---

### Task 10: Final cleanup and verification

**Files:**
- Verify: all test files
- Modify: `src/processors/generators/mod.rs` (remove dead macro if not done)
- Modify: `src/processors/mod.rs` (clean up any unused imports)

- [ ] **Step 1: Run full test suite**

Run: `cargo test 2>&1 | tail -10`
Expected: All 340 tests pass

- [ ] **Step 2: Run clippy**

Run: `cargo clippy 2>&1 | tail -20`
Expected: No warnings

- [ ] **Step 3: Build release**

Run: `cargo build --release 2>&1`
Expected: Compiles successfully

- [ ] **Step 4: Verify teaching-slides still works**

Run (from teaching-slides dir): `rsconstruct build -j 20 --dry-run 2>&1 | tail -5`
Expected: Products discovered correctly

- [ ] **Step 5: Commit any cleanup**

```bash
git add -A
git commit -m "refactor: final cleanup after ProcessorBase migration"
```
