# Example: Debug Compilation and Transpilation

This example demonstrates debugging the entire compilation pipeline: from Python source code through transpilation (Depyler), compilation (rustc), to final binary execution. Perfect for diagnosing issues like DEPYLER-0435.

---

## Overview: The Full Pipeline

When transpiling Python to Rust with Depyler, we have multiple layers:

```
Python Source (.py)
    ↓ [Depyler Transpiler]
Generated Rust (.rs) + Source Map (.map)
    ↓ [cargo/rustc Compiler]
Binary Executable + DWARF Debug Info
    ↓ [Execution]
Runtime Syscalls
```

**Renacer can trace and correlate across ALL these layers.**

---

## Scenario 1: Debugging Slow Compilation

Your Depyler-transpiled project takes 15 minutes to compile. Let's find why.

### Step 1: Measure Compilation Performance

```bash
# Baseline: How long does it take?
$ time cargo build --release

real    15m23s
user    42m15s
sys     2m34s
```

**Problem:** 15 minutes is too slow. What's happening?

### Step 2: Trace the Build Process

```bash
$ renacer -f -c -e 'trace=file,process' -- cargo build --release
```

**Output (summary):**

```
System Call Summary (Multi-Process):
=====================================
Process Tree:
[PID 12345] cargo build
  └─ [PID 12346] rustc --crate-name myapp ...
       ├─ [PID 12347] rustc --crate-name dep1 ...
       ├─ [PID 12348] rustc --crate-name dep2 ...
       └─ [PID 12349] ld (linker)

Syscall Statistics:
Syscall    Total Calls   Total Time    Process Distribution
openat     125,847       45.2s         rustc: 98%, ld: 2%
read       2,456,789     23.4s         rustc: 100%
write      345,678       12.3s         rustc: 95%, ld: 5%
execve     1,247         8.9s          cargo: 100%
```

**Analysis:**
- 125,847 `openat` calls taking 45 seconds
- 1,247 `execve` calls (many rustc invocations)
- Most time in `rustc` reading files

### Step 3: Find What's Being Opened

```bash
$ renacer -f -e 'trace=openat' -- cargo build --release 2>&1 | \
  grep -o '"/[^"]*"' | sort | uniq -c | sort -rn | head -20
```

**Output:**

```
12,456 "/home/user/.cargo/registry/cache/..."
8,923  "/usr/lib/rustlib/x86_64-unknown-linux-gnu/..."
4,567  "/home/user/project/target/release/deps/..."
3,421  "/home/user/project/src/generated/mod_0001.rs"
3,420  "/home/user/project/src/generated/mod_0002.rs"
3,419  "/home/user/project/src/generated/mod_0003.rs"
... (1,200 more generated modules)
```

**Root Cause Found:** Depyler generated 1,200+ separate `.rs` modules, each opened ~3 times during compilation!

### Step 4: Correlate with Source Code

```bash
# Find where rustc is spending time
$ renacer -f --source -e 'trace=read' -- cargo build --release 2>&1 | \
  grep "rustc" | head -10
```

**Output:**

```
[PID 12346] read(3, "// Generated by Depyler\nmod mo...", 8192) = 8192
  [/rustc/hash/compiler/rustc_parse/src/parser/item.rs:234 in parse_item]

[PID 12346] read(4, "// Generated by Depyler\npub fn...", 8192) = 8192
  [/rustc/hash/compiler/rustc_parse/src/parser/item.rs:234 in parse_item]
```

**Insight:** `rustc` parser is reading each generated file individually.

### Step 5: Analyze Depyler Output Structure

```bash
# Trace the transpilation step
$ renacer -e 'trace=file' -- depyler transpile main.py --output src/generated/
```

**Output:**

```
openat(AT_FDCWD, "main.py", O_RDONLY) = 3
# ... parsing ...
openat(AT_FDCWD, "src/generated/mod_0001.rs", O_WRONLY|O_CREAT) = 4
write(4, "// Generated by Depyler\n...", 2456) = 2456
close(4) = 0
openat(AT_FDCWD, "src/generated/mod_0002.rs", O_WRONLY|O_CREAT) = 4
write(4, "// Generated by Depyler\n...", 2398) = 2398
close(4) = 0
# ... 1,198 more files ...
```

**Problem:** Depyler is generating 1,200 small modules instead of consolidating them.

**Solution:** Configure Depyler to use `--merge-modules` flag:

```bash
$ depyler transpile main.py --output src/generated/ --merge-modules=100
```

**Result:** 1,200 modules → 12 modules (100 items per file)

### Step 6: Verify Improvement

```bash
$ time cargo build --release

real    2m34s   ← 6x faster!
user    7m12s
sys     0m23s
```

**Improvement:**
- Compilation time: 15m23s → 2m34s (6x faster)
- `openat` calls: 125,847 → 12,456 (10x reduction)

---

## Scenario 2: Finding Missing Dependencies

Your transpiled code fails to compile with cryptic error. Let's trace it.

### Problem Statement

```bash
$ cargo build
   Compiling myapp v0.1.0
error[E0433]: failed to resolve: use of undeclared crate or module `std`
  --> src/generated/mod_main.rs:42:5
   |
42 |     std::fs::read_to_string(path)
   |     ^^^ use of undeclared crate or module `std`
```

**Question:** Why can't it find `std`? It should be automatically available!

### Step 1: Trace the Failing Compilation

```bash
$ renacer -f --source -e 'trace=openat' -- cargo build 2>&1 | \
  grep -E "\.rs|ENOENT"
```

**Output:**

```
[PID 12350] openat(AT_FDCWD, "src/generated/mod_main.rs", O_RDONLY) = 3
[PID 12350] openat(AT_FDCWD, "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd.rlib", O_RDONLY) = 4
[PID 12350] openat(AT_FDCWD, "src/generated/prelude.rs", O_RDONLY) = -2 (ENOENT)
[PID 12350] openat(AT_FDCWD, "src/lib.rs", O_RDONLY) = -2 (ENOENT)
```

**Analysis:**
- `rustc` successfully opens `libstd.rlib` (so std EXISTS)
- Looking for `prelude.rs` but not found (ENOENT)
- The generated code is missing the prelude import!

### Step 2: Check Generated Code

```bash
$ head -20 src/generated/mod_main.rs
```

**Output:**

```rust
// Generated by Depyler v0.4.0
// Source: main.py

#![no_std]  ← PROBLEM! This disables std library

pub fn read_file(path: &str) -> String {
    std::fs::read_to_string(path)  // ← Tries to use std, but no_std is set!
}
```

**Root Cause:** Depyler incorrectly set `#![no_std]` attribute in generated code.

### Step 3: Trace Depyler Transpilation

```bash
$ renacer --source -e 'trace=write' -- depyler transpile main.py 2>&1 | \
  grep "no_std"
```

**Output:**

```
write(4, "#![no_std]\n", 11) = 11
  [depyler/src/codegen/rust_emitter.rs:156 in emit_crate_attrs]
```

**Found It:** `depyler/src/codegen/rust_emitter.rs:156` is emitting `#![no_std]` incorrectly.

### Step 4: Fix and Verify

Edit Depyler configuration:

```toml
# depyler.toml
[codegen]
use_std = true  # ← Force std library usage
no_std = false
```

**Recompile:**

```bash
$ depyler transpile main.py
$ cargo build
   Compiling myapp v0.1.0
   Finished dev [unoptimized + debuginfo] target(s) in 12.3s
```

**Success!**

---

## Scenario 3: Transpiler Source Mapping (Sprint 24-28)

A runtime error occurs in transpiled code. Can we trace it back to the original Python?

### Problem: Runtime Panic in Generated Rust

```bash
$ ./target/release/myapp
thread 'main' panicked at 'index out of bounds: the len is 5 but the index is 10',
  src/generated/mod_array_ops.rs:234:9
```

**Question:** What line in the **original Python** caused this?

### Step 1: Trace Runtime Execution

```bash
$ renacer --source -- ./target/release/myapp
```

**Output:**

```
read(3, "[1, 2, 3, 4, 5]", 4096) = 15
  [src/generated/mod_array_ops.rs:230 in parse_array]

read(3, "10", 4096) = 2
  [src/generated/mod_array_ops.rs:232 in read_index]

# PANIC at src/generated/mod_array_ops.rs:234
```

**DWARF tells us:** Crash is at `mod_array_ops.rs:234`

**But we need:** Original Python source location!

### Step 2: Use Transpiler Source Map

```bash
$ renacer --source --transpiler-map src/generated.map -- ./target/release/myapp
```

**Output (with source map):**

```
read(3, "[1, 2, 3, 4, 5]", 4096) = 15
  [src/generated/mod_array_ops.rs:230 in parse_array]
  ↳ [Original: main.py:42 in process_data]  ← Transpiler mapping!

read(3, "10", 4096) = 2
  [src/generated/mod_array_ops.rs:232 in read_index]
  ↳ [Original: main.py:43 in process_data]

# PANIC at src/generated/mod_array_ops.rs:234
  ↳ [Original: main.py:44 in process_data]  ← Root cause in Python!
```

**Correlation Chain:**
```
Python: main.py:44
   ↓ (transpiled to)
Rust: src/generated/mod_array_ops.rs:234
   ↓ (compiled to)
Binary: 0x401234 (with DWARF)
   ↓ (executes)
Syscall: read(3, ...) = 2
```

### Step 3: Inspect Original Python Source

```python
# main.py:42-44 (the actual bug)
def process_data(filename):
    data = json.load(open(filename))  # Line 42
    index = int(input("Enter index: "))  # Line 43
    return data[index]  # Line 44 ← Out of bounds!
```

**Root Cause Found:** Python code doesn't validate `index` before accessing `data[index]`.

**Fix:**

```python
def process_data(filename):
    data = json.load(open(filename))
    index = int(input("Enter index: "))
    if index < 0 or index >= len(data):
        raise IndexError(f"Index {index} out of range [0, {len(data)})")
    return data[index]
```

---

## Scenario 4: Multi-Process Compilation Pipeline

Understanding the full build dependency tree.

### Step 1: Trace Full Build with Process Tree

```bash
$ renacer -f -c --function-time -- cargo build
```

**Output:**

```
Process Hierarchy:
==================
[PID 10001] cargo build (parent)
    Time in execve: 234ms (spawning children)
    Time in wait4: 145.2s (waiting for rustc)

  [PID 10002] rustc --crate-name myapp (child 1)
      Time in openat: 12.3s (reading source files)
      Time in read: 45.6s (parsing)
      Time in write: 8.9s (emitting LLVM IR)
      Time in execve: 1.2s (spawning linker)

    [PID 10003] ld (linker, grandchild)
        Time in openat: 3.4s (reading .o files)
        Time in read: 8.9s (linking)
        Time in write: 2.1s (writing binary)

  [PID 10004] rustc --crate-name dep1 (child 2 - parallel)
      Time in openat: 5.6s
      Time in read: 23.4s
      Time in write: 4.3s

  [PID 10005] rustc --crate-name dep2 (child 3 - parallel)
      Time in openat: 4.2s
      Time in read: 18.9s
      Time in write: 3.1s
```

**Analysis:**
- `cargo` spends 145s waiting for child processes
- 3 `rustc` processes run in parallel
- Each `rustc` spends most time in `read` (parsing)
- Linker (`ld`) is relatively fast (14.4s total)

### Step 2: Find Slowest Compilation Unit

```bash
$ renacer -f -c -e 'trace=file' -- cargo build 2>&1 | \
  grep -A 3 "rustc.*myapp"
```

**Output:**

```
[PID 10002] rustc --crate-name myapp
  Syscall Statistics:
  openat: 8,923 calls, 12.3s total
  read:   125,847 calls, 45.6s total  ← Slowest operation
  write:  34,521 calls, 8.9s total
```

**Bottleneck:** `rustc` reading 125,847 times (parsing generated code)

### Step 3: Optimize Build Parallelism

```bash
# Before: Serial compilation
$ time cargo build -j 1
real    8m34s

# After: Parallel compilation
$ time cargo build -j 4
real    2m45s  ← 3x faster
```

**Result:** Multi-process tracing revealed parallelization opportunity.

---

## Scenario 5: Debugging Compilation Errors

The compiler fails with an error. Where is the problem in the pipeline?

### Problem: Mysterious Type Error

```bash
$ cargo build
error[E0308]: mismatched types
  --> src/generated/mod_types.rs:89:5
   |
89 |     result
   |     ^^^^^^ expected `i32`, found `String`
```

### Step 1: Trace Code Generation

```bash
$ renacer --transpiler-map depyler.map -- depyler transpile types.py
```

**Output:**

```
write(4, "fn convert(x: String) -> i32 {\n", 31) = 31
  [depyler/src/codegen/type_inference.rs:123 in emit_function]
  ↳ [Original: types.py:12 in convert]

write(4, "    result\n", 11) = 11
  [depyler/src/codegen/type_inference.rs:145 in emit_return]
  ↳ [Original: types.py:15 in convert]  ← Wrong type inferred!
```

**Root Cause:** Depyler's type inference at `type_inference.rs:123` incorrectly assumed `String` when it should be `i32`.

### Step 2: Check Original Python Type Hint

```python
# types.py:12-15
def convert(x: str) -> int:  # ← Type hint says int
    result = int(x)  # Line 14
    return result  # Line 15 ← Should return int, not str
```

**Problem:** Depyler didn't respect Python type hints during code generation.

### Step 3: Fix Depyler Type Mapping

Report bug to Depyler:
- Type hint `int` should map to Rust `i32`
- Type inference failed to see `int(x)` converts to integer

---

## Advanced: Full Stack Correlation

Combining all features to trace from Python source to binary execution.

### Scenario: Performance Regression After Transpilation

Python version runs fast, but transpiled Rust version is slow. Why?

### Step 1: Compare Python vs Transpiled Performance

```bash
# Python baseline
$ time python3 main.py
real    0.5s

# Transpiled Rust version
$ time ./target/release/myapp
real    5.2s  ← 10x slower!
```

### Step 2: Profile Both Versions

```bash
# Profile Python
$ renacer -c -e 'trace=file' -- python3 main.py
```

**Output (Python):**

```
Syscall Statistics:
openat: 12 calls, 23ms
read:   156 calls, 45ms
write:  89 calls, 12ms
Total:  80ms (rest is Python interpreter overhead)
```

```bash
# Profile Transpiled Rust
$ renacer -c -e 'trace=file' -- ./target/release/myapp
```

**Output (Rust):**

```
Syscall Statistics:
openat: 1,247 calls, 2.3s  ← 100x more opens!
read:   156 calls, 45ms
write:  89 calls, 12ms
Total:  2.4s
```

**Problem:** Rust version makes 1,247 `openat` calls vs 12 in Python!

### Step 3: Find Source of Extra Opens

```bash
$ renacer --source --transpiler-map depyler.map -e 'trace=openat' -- ./target/release/myapp 2>&1 | head -50
```

**Output:**

```
openat(AT_FDCWD, "config.json", O_RDONLY) = 3
  [src/generated/mod_config.rs:45 in load_config]
  ↳ [Original: main.py:23 in initialize]

openat(AT_FDCWD, "config.json", O_RDONLY) = 3
  [src/generated/mod_config.rs:45 in load_config]
  ↳ [Original: main.py:23 in initialize]  ← Same file opened again!

# ... 1,245 more identical opens ...
```

**Root Cause:** Transpiled code calls `load_config()` in a loop without caching!

### Step 4: Check Original Python

```python
# main.py:22-30
def initialize():
    for i in range(1000):
        config = load_config()  # Line 23 ← Opens file every iteration!
        process_item(i, config)
```

**Problem:** Python's file caching masked this issue, but Rust opens file every time.

**Fix:**

```python
def initialize():
    config = load_config()  # ← Cache outside loop
    for i in range(1000):
        process_item(i, config)
```

**Result After Fix:**

```bash
$ time ./target/release/myapp
real    0.4s  ← 13x faster! Even faster than Python!
```

---

## Debugging Workflow Summary

### Full Pipeline Debugging Process:

```bash
# 1. Trace transpilation (Depyler)
$ renacer -e 'trace=file' -- depyler transpile main.py

# 2. Trace compilation (cargo/rustc)
$ renacer -f -c -e 'trace=file,process' -- cargo build

# 3. Trace execution with source maps
$ renacer --source --transpiler-map out.map -- ./myapp

# 4. Export for analysis
$ renacer --format json --transpiler-map out.map -- ./myapp > trace.json
```

### Key Techniques:

1. **`-f` (follow forks)** - Trace multi-process builds
2. **`--source` (DWARF)** - Correlate binary → Rust source
3. **`--transpiler-map`** - Correlate Rust → Python source
4. **`-e trace=file,process`** - Focus on build-relevant syscalls
5. **`-c` (statistics)** - Measure compilation performance

---

## Common Compilation Issues

### Issue 1: Missing Source Maps

**Symptom:**

```bash
$ renacer --transpiler-map out.map -- ./myapp
Error: Transpiler map not found: out.map
```

**Solution:**

```bash
# Ensure Depyler generates source maps
$ depyler transpile main.py --emit-source-map
# Creates: src/generated.map
```

### Issue 2: DWARF Info Stripped

**Symptom:**

```bash
$ renacer --source -- ./target/release/myapp
Warning: No DWARF debug info found
```

**Solution:**

```bash
# Always compile with debug info
$ cargo build --release  # Includes debug info by default
# OR explicitly:
$ RUSTFLAGS="-C debuginfo=2" cargo build --release
```

### Issue 3: Too Many Processes to Track

**Symptom:**

```bash
$ renacer -f -- cargo build
Error: Process limit exceeded (ptrace: ENOMEM)
```

**Solution:**

```bash
# Limit parallel jobs
$ renacer -f -- cargo build -j 4

# OR filter to specific crate
$ renacer -f -- cargo build -p myapp
```

---

## Best Practices

### 1. Always Use Source Maps for Transpiled Code

```bash
# Generate with source maps
$ depyler transpile --emit-source-map main.py

# Trace with source maps
$ renacer --transpiler-map out.map -- ./myapp
```

### 2. Profile Before Optimizing

```bash
# Baseline first
$ renacer -c -- cargo build > before.txt

# Optimize, then compare
$ renacer -c -- cargo build > after.txt
$ diff before.txt after.txt
```

### 3. Use Multi-Process Tracing for Builds

```bash
# Always use -f for build systems
$ renacer -f -c -- cargo build
$ renacer -f -c -- make
```

### 4. Export for CI/CD Integration

```bash
# Export compilation stats for regression detection
$ renacer -f --format json -c -- cargo build > build-trace.json

# CI script checks:
# - Total compilation time < threshold
# - No excessive file opens during linking
# - Process tree depth reasonable
```

---

## Related

- [Multi-Process Tracing](./multi-process.md) - Using `-f` flag
- [DWARF Source Correlation](../core-concepts/dwarf-correlation.md) - `--source` flag
- [Export to JSON/CSV](./export-data.md) - Automated analysis
- [Debug Performance Issues](./debug-performance.md) - General profiling
