Introduction

towl is a fast command-line tool built in Rust that scans codebases for TODO comments. It provides an interactive TUI for browsing and managing TODOs, can create GitHub issues from them, and supports multiple output formats for CI/scripting. It detects TODO, FIXME, HACK, NOTE, and BUG comments across many languages, with configurable patterns, context-aware output, and robust resource limits.

Key Features

• Interactive TUI -- Browse, filter, sort, and peek at TODOs in a full-screen terminal interface powered by ratatui
• GitHub integration -- Create GitHub issues from selected TODOs and automatically replace comments with issue links
• Multi-language support -- Scans Rust, Python, JavaScript, Go, Shell, and more via configurable comment prefixes and function patterns
• Multiple output formats -- JSON, CSV, Markdown, TOML, and terminal table (non-interactive mode)
• Type filtering & sorting -- Filter results by TODO type; sort by file, line, type, or priority
• Context-aware -- Captures surrounding code lines and enclosing function names
• Configurable -- Customise file extensions, exclude patterns, comment prefixes, and TODO patterns via .towl.toml (override with --config or TOWL_CONFIG env var)
• AI validation -- LLM-powered TODO analysis using Claude, OpenAI, or local CLI agents (Claude Code, Codex)
• Safe by design -- Path traversal protection, resource limits, symlink resolution, and secret handling for tokens
• Fast -- Concurrent file scanning, async I/O with tokio, compiled regex patterns, and static enum dispatch

How It Works

                ┌──────────┐
                │  Config   │  --config / TOWL_CONFIG / .towl.toml + env vars
                └────┬─────┘
                     │
                ┌────▼─────┐
                │  Scanner  │  Walks directory tree, scans files concurrently
                └────┬─────┘
                     │
                ┌────▼─────┐
                │  Parser   │  Matches comment prefixes + TODO patterns
                └────┬─────┘
                     │
                ┌────▼─────┐
                │  LLM      │  --ai: validates TODOs with AI (optional)
                └────┬─────┘
                     │
              ┌──────┴──────┐
              │             │
        ┌─────▼────┐  ┌────▼─────┐
        │   TUI     │  │  Output   │  Non-interactive: formats + writes
        │ (default) │  │  (-N)     │
        └─────┬────┘  └──────────┘
              │
        ┌─────▼────┐
        │ Processor │  Replaces TODOs with GitHub issue links
        └──────────┘

1. Config loads settings from .towl.toml (or a custom path via --config / TOWL_CONFIG), merges environment variables for GitHub and LLM integration
2. Scanner walks the directory tree using the ignore crate, scanning matching files concurrently with bounded parallelism
3. Parser reads each file, matches comment prefixes and TODO patterns via compiled regex, extracts context lines and function names
4. LLM (optional, --ai) validates each TODO with an AI model, classifying them as Valid, Invalid, or Uncertain
5. TUI (default) presents an interactive interface for browsing, filtering, and selecting TODOs to create as GitHub issues
6. Output (non-interactive) formats the collected TodoComment items into the requested format and writes to a file or stdout
7. Processor replaces TODO comments in source files with GitHub issue links after issues are created

Quick Example

# Scan current directory (opens interactive TUI)
towl scan

# Non-interactive: output as terminal table
towl scan -N

# Non-interactive: output to JSON file
towl scan -N -f json -o todos.json

# Filter to only FIXME comments
towl scan -N -t fixme

# AI analysis: validate TODOs and filter out invalid ones
towl scan -N --ai

# Create GitHub issues
towl scan -N -g

# Show current configuration
towl config

Installation

From crates.io

cargo install towl

Requires Rust 1.75 or later. Install Rust via rustup if needed.

From Source

git clone https://github.com/glottologist/towl.git
cd towl
cargo build --release

The binary will be at target/release/towl.

Verify Installation

towl --version
towl --help

Requirements

• Rust: 1.75+
• git: Required on PATH for towl init (extracts GitHub owner/repo from the git remote)
• OS: Linux, macOS, Windows

Quick Start

1. Initialise Configuration

Inside a git repository with a GitHub remote:

towl init

This creates .towl.toml with sensible defaults. GitHub owner/repo are auto-detected from git remote get-url origin at runtime (not stored in the config file).

If .towl.toml already exists, use --force to overwrite:

towl init --force

2. Scan for TODOs (Interactive)

# Scan the current directory (opens TUI)
towl scan

# Scan a specific path
towl scan src/

# Use a config file from a custom location
towl scan -c .config/.towl.toml

The interactive TUI lets you browse, filter, sort, peek at source code, and create GitHub issues from selected TODOs.

3. Scan for TODOs (Non-Interactive)

Use --non-interactive / -N for CI pipelines and scripting:

# Terminal table output
towl scan -N

# Enable verbose output (file counts, timing)
towl scan -N -v

4. Choose an Output Format

Non-interactive mode supports multiple output formats:

# Terminal table (default)
towl scan -N

# JSON file
towl scan -N -f json -o todos.json

# CSV file
towl scan -N -f csv -o todos.csv

# Markdown file
towl scan -N -f markdown -o todos.md

# TOML file
towl scan -N -f toml -o todos.toml

Note: File-based formats (json, csv, toml, markdown) require the -o flag with a matching file extension. Terminal/table formats always output to stdout.

5. Filter by Type

# Only TODO comments
towl scan -N -t todo

# Only FIXME comments
towl scan -N -t fixme

# Only BUG comments
towl scan -N -t bug

Available types: todo, fixme, hack, note, bug

6. Create GitHub Issues

Set your GitHub token:

export TOWL_GITHUB_TOKEN=ghp_your_token_here

Then create issues from TODOs:

# Create GitHub issues (non-interactive)
towl scan -N -g

# Preview issues without creating them
towl scan -N -g -n

In interactive mode, select TODOs with Space and press Enter to create issues.

7. View Configuration

towl config

# From a custom config path
towl config -c .config/.towl.toml

Displays a tree view of all active settings including file extensions, exclude patterns, comment prefixes, TODO patterns, and GitHub configuration.

You can also set the TOWL_CONFIG environment variable to avoid passing --config every time:

export TOWL_CONFIG=.config/.towl.toml
towl scan

Configuration

towl uses a .towl.toml file in the project root for configuration. All fields have sensible defaults -- you only need to override what you want to change.

You can point to a config file in a different location using:

• --config / -c flag on scan and config commands
• TOWL_CONFIG environment variable

The --config flag takes precedence over TOWL_CONFIG, which takes precedence over the default .towl.toml.

Config File

Create .towl.toml manually or run towl init:

[parsing]
file_extensions = ["rs", "toml", "json", "yaml", "yml", "sh", "bash"]
exclude_patterns = ["target/*", ".git/*"]
include_context_lines = 10

Parsing Section

Default TODO Patterns

todo_patterns = [
    "(?i)\\bTODO:\\s*(.*)",
    "(?i)\\bFIXME:\\s*(.*)",
    "(?i)\\bHACK:\\s*(.*)",
    "(?i)\\bNOTE:\\s*(.*)",
    "(?i)\\bBUG:\\s*(.*)",
]

All patterns are case-insensitive by default. Each pattern must contain a capture group (.*) for extracting the description text.

Default Function Patterns

function_patterns = [
    "^\\s*(pub\\s+)?fn\\s+(\\w+)",            # Rust
    "^\\s*def\\s+(\\w+)",                      # Python
    "^\\s*(async\\s+)?function\\s+(\\w+)",     # JavaScript
    "^\\s*(public|private|protected)?\\s*(static\\s+)?\\w+\\s+(\\w+)\\s*\\(",  # Java/C#
    "^\\s*func\\s+(\\w+)",                     # Go/Swift
]

Pattern Limits

Each pattern field is limited to 100 entries. Individual regex patterns are limited to 256 characters. Config string values (e.g., owner, repo) are limited to 512 characters. These limits prevent denial-of-service via malicious configuration files.

GitHub Section

Owner and repo are always auto-detected from git remote get-url origin at runtime -- they are not stored in the config file. Use TOWL_GITHUB_OWNER and TOWL_GITHUB_REPO environment variables to override if needed.

Note: The GitHub token is never stored in the config file. Use the TOWL_GITHUB_TOKEN environment variable.

LLM Section

Note: The LLM API key is never stored in the config file. Use the TOWL_LLM_API_KEY environment variable. See AI Analysis for usage details.

Environment Variables

Eight environment variables override defaults:

Config Loading Order

1. Built-in defaults
2. Config file resolved as: --config flag > TOWL_CONFIG env var > .towl.toml
3. Git remote auto-detection for owner/repo
4. Environment variable overrides (TOWL_GITHUB_*, TOWL_LLM_*)

If no config file exists at the resolved path, defaults are used without error.

Viewing Active Configuration

# Show config from default .towl.toml
towl config

# Show config from a custom path
towl config -c .config/.towl.toml

Example output:

📋 Towl Configuration
┌─ Parsing
│  ├─ File Extensions: bash, json, rs, sh, toml, yaml, yml
│  ├─ Exclude Patterns: target/*, .git/*
│  ├─ Context Lines: 10
│  ├─ Comment Prefixes:
│  │  ├─ //
│  │  ├─ ^\s*#
│  │  ├─ /\*
│  │  └─ ^\s*\*
│  ├─ TODO Patterns:
│  │  ├─ (?i)\bTODO:\s*(.*)
│  │  ...
│  └─ Function Patterns:
│     ├─ ^\s*(pub\s+)?fn\s+(\w+)
│     ...
└─ GitHub
   ├─ Owner: glottologist
   ├─ Repo: towl
   └─ Token: not set

Scanning for TODOs

The towl scan command walks a directory tree, reads each matching file, and extracts TODO-style comments using compiled regex patterns.

Config Override

By default, towl reads .towl.toml from the project root. Use --config / -c to load from a different path:

towl scan -c .config/.towl.toml
towl scan -c .config/.towl.toml src/

You can also set the TOWL_CONFIG environment variable. The --config flag takes precedence over the env var.

Interactive Mode (Default)

By default, towl scan opens an interactive TUI:

towl scan
towl scan src/

See Interactive TUI for details on the TUI interface.

Non-Interactive Mode

Use --non-interactive / -N to disable the TUI (for CI/scripting):

towl scan -N
towl scan -N src/
towl scan -N -v

How Scanning Works

1. Directory walk -- Uses the ignore crate to traverse the file tree, respecting .gitignore rules automatically
2. Extension filter -- Only files matching file_extensions in config are read (default: rs, toml, json, yaml, yml, sh, bash)
3. Exclude patterns -- Files matching exclude_patterns are skipped (default: target/*, .git/*)
4. Concurrent scanning -- Matching files are scanned concurrently with bounded parallelism (up to 64 files at once)
5. Content parsing -- Each file is read and scanned for lines matching comment_prefixes, then checked against todo_patterns
6. Context extraction -- Surrounding lines and enclosing function names are captured

Verbose Mode

The -v / --verbose flag prints scan metrics to stderr (non-interactive mode only):

towl scan -N -v

Files scanned: 42
Files skipped: 3
Files errored: 0
Scan duration: 12ms

Filtering by Type

Restrict results to a single TODO type:

towl scan -N -t todo      # Only TODO comments
towl scan -N -t fixme     # Only FIXME comments
towl scan -N -t hack      # Only HACK comments
towl scan -N -t note      # Only NOTE comments
towl scan -N -t bug       # Only BUG comments

The filter value is case-insensitive on the command line but stored lowercase internally.

GitHub Issue Creation

Create GitHub issues from found TODOs:

# Create issues
towl scan -N -g

# Preview without creating
towl scan -N -g -n

When issues are created, towl automatically replaces the TODO comment in source files with a link to the created issue. Duplicate detection prevents creating issues for TODOs that already have a matching open issue.

In interactive mode, select TODOs with Space and press Enter to create issues.

AI Analysis

Use the --ai flag to validate TODOs with an LLM:

# Analyse and filter out invalid TODOs
towl scan -N --ai

# Interactive mode with AI analysis
towl scan --ai

# Create GitHub issues for valid TODOs only (enriched with AI reasoning)
towl scan -N --ai -g

In non-interactive mode, TODOs classified as Invalid are automatically excluded from the output. See AI Analysis for full details.

Combining Options

Options compose freely:

# Scan src/, output FIXME comments as JSON to a file, verbose
towl scan -N src/ -t fixme -f json -o fixmes.json -v

# Scan and create GitHub issues for TODO comments only
towl scan -N -t todo -g

# AI-validated FIXMEs as JSON
towl scan -N --ai -t fixme -f json -o fixmes.json

# Use a custom config for everything
towl scan -c .config/.towl.toml -N src/ -t fixme -f json -o fixmes.json

Resource Limits

towl enforces hard limits to prevent runaway scans:

When a limit is hit, scanning stops gracefully and returns the results collected so far.

Scan Result

The scan produces a ScanResult containing:

• todos -- The list of extracted TodoComment items
• files_scanned -- Number of files successfully read
• files_skipped -- Number of files skipped (wrong extension, excluded, too large)
• files_errored -- Number of files that failed to read (permissions, encoding)
• duration -- Wall-clock time for the scan

Two convenience checks:

• all_files_failed() -- Returns true when no files were scanned but errors occurred (likely a permissions or path issue)
• is_clean() -- Returns true when zero TODOs were found and zero files errored

Path Safety

• Path traversal -- Paths containing .. components are rejected
• Symlink resolution -- Symlinks are resolved before processing to prevent escape from the scan root
• .gitignore -- Respected automatically via the ignore crate

Interactive TUI

By default, towl scan opens an interactive terminal interface powered by ratatui. The TUI lets you browse, filter, sort, and peek at TODOs, then create GitHub issues from selected items.

Launching

# Opens TUI with TODOs from current directory
towl scan

# Opens TUI with TODOs from a specific path
towl scan src/

To bypass the TUI (for CI/scripting), use --non-interactive / -N.

Modes

The TUI has six modes:

Browse

The main view. Displays all TODOs in a scrollable list with type, description, file path, and line number.

When launched with --ai, each row shows a validity indicator (V/I/?) and is colour-coded: green for valid, red for invalid, yellow for uncertain.

Peek

Shows the source code surrounding the selected TODO with syntax context. The TODO line is highlighted. When --ai is active, the AI Analysis section is displayed below the source code with the validity, confidence score, and reasoning. The reasoning text word-wraps to fit the popup width.

Confirm

Appears after pressing Enter in browse mode with selected TODOs. Shows a summary of the TODOs that will be created as GitHub issues.

Creating

Displays a progress view while GitHub issues are being created. Shows the current phase (initialising client, loading existing issues, creating issues, replacing TODOs in files) and a progress counter.

No keyboard input is accepted during creation (except Ctrl+C to force quit).

Done

Shows the results after issue creation completes -- number of issues created, any errors encountered. Press q, Esc, or Enter to exit.

Delete Confirm (requires --ai)

Appears after pressing d in Browse mode with selected invalid TODOs. Lists the TODOs that will be removed from source files.

Only TODOs marked as Invalid by the AI are eligible for deletion.

Workflow

1. Run towl scan to open the TUI (with --ai, a progress bar shows during analysis)
2. Browse the TODO list -- use f to filter by type, s/r to sort
3. Press p to peek at source code around a TODO
4. Select TODOs with Space (or a to select all visible)
5. Press Enter to review selected TODOs
6. Press y to create GitHub issues
7. towl creates the issues, skips duplicates, and replaces TODO comments with issue links in source files

AI Analysis

towl can use an LLM (Claude or any OpenAI-compatible model) to validate whether each TODO is still relevant. The --ai flag triggers analysis that determines if a TODO is Valid, Invalid, or Uncertain.

Setup

Set your API key as an environment variable:

# Claude (default)
export TOWL_LLM_API_KEY=sk-ant-your-key-here

# Or for OpenAI
export TOWL_LLM_API_KEY=sk-your-openai-key
export TOWL_LLM_PROVIDER=openai

The API key is stored as a SecretString and never written to config files or logs.

Basic Usage

# Non-interactive: analyse and filter out invalid TODOs
towl scan -N --ai

# Interactive: analyse and show results in TUI
towl scan --ai

# Combine with other flags
towl scan -N --ai -t fixme -f json -o fixmes.json
towl scan -N --ai -g  # create GitHub issues for valid TODOs only

How It Works

For each TODO, the LLM receives:

1. TODO description -- the comment text
2. Expanded context -- ~30 lines of surrounding source code
3. Function body -- the complete enclosing function (if detected)

The LLM determines:

• Is it resolved? -- Does the code already do what the TODO asks?
• Is it relevant? -- Does the code/feature still exist?
• Is it actionable? -- Is the TODO clear and specific?

Based on these checks, each TODO is classified as Valid, Invalid, or Uncertain with a confidence score (0-100%).

Non-Interactive Mode

With -N --ai, invalid TODOs are automatically filtered out of the results:

towl scan -N --ai
# Only valid and uncertain TODOs appear in output

towl scan -N --ai -g
# GitHub issues created only for valid TODOs, enriched with AI reasoning

Interactive Mode (TUI)

With --ai (no -N), a progress bar is displayed while TODOs are being analysed:

  Analysing TODOs [████████████░░░░░░░░░░░░░░░░░░] 12/30

Once analysis completes, the TUI launches with results:

• Validity column -- Each TODO shows V (Valid), I (Invalid), or ? (Uncertain)
• Colour coding -- Green for valid, red for invalid, yellow for uncertain
• Peek view -- Press p to see the LLM's reasoning below the source code (text wraps to fit the popup width)
• Delete invalid TODOs -- Select invalid TODOs and press d to remove them from source files (with confirmation)

Delete Workflow

1. Select invalid TODOs with Space (or a to select all visible)
2. Press d to open the delete confirmation dialog
3. Review the list of TODOs that will be removed
4. Press y to confirm deletion, or n to cancel
5. towl removes the comment lines from source files using atomic writes

Note: Only TODOs marked as Invalid by the AI can be deleted via d. Valid and Uncertain TODOs are excluded from deletion.

GitHub Issue Enrichment

When creating GitHub issues (either with -g or via the TUI), valid TODOs include an AI Analysis section in the issue body:

## AI Analysis

**Validity:** Valid
**Confidence:** 92%

### Reasoning

The caching layer referenced in this TODO has not been implemented.
The function currently makes direct database calls on every request.

### Enhanced Description

This TODO identifies a performance bottleneck where database queries
are executed on every request without caching. Adding a caching layer
would reduce database load and improve response times.

Configuration

Add a [llm] section to .towl.toml:

[llm]
provider = "claude"                      # "claude" or "openai"
model = "claude-opus-4-6"             # model identifier
# base_url = "http://localhost:11434/v1"  # for Ollama/vLLM
max_concurrent_analyses = 5              # concurrent LLM requests
max_analyse_count = 50                   # max TODOs to analyse per scan
max_tokens = 4096                        # LLM response token limit

Environment Variables

Using Claude Code or Codex CLI

If you have claude (Claude Code) or codex (OpenAI Codex CLI) installed, you can use them directly without an API key:

# Use Claude Code CLI
export TOWL_LLM_PROVIDER=claude-code
towl scan --ai

# Use Codex CLI
export TOWL_LLM_PROVIDER=codex
towl scan --ai

Or set in .towl.toml:

[llm]
provider = "claude-code"   # or "codex"
# command = "/custom/path/to/claude"   # optional override
# args = ["-p", "--output-format", "json"]  # optional override

No TOWL_LLM_API_KEY is needed -- the CLI agents manage their own authentication.

Auto-fallback: If the CLI binary is not found on PATH, towl automatically falls back to the corresponding API provider (claude-code -> Claude API, codex -> OpenAI API). The API fallback requires TOWL_LLM_API_KEY to be set.

Using with Ollama or Local Models

export TOWL_LLM_PROVIDER=openai
export TOWL_LLM_MODEL=llama3
export TOWL_LLM_BASE_URL=http://localhost:11434/v1
export TOWL_LLM_API_KEY=ollama  # Ollama doesn't need a real key

towl scan -N --ai

Rate Limiting

Two configurable limits prevent excessive API usage:

When the TODO count exceeds max_analyse_count, only the first N TODOs are analysed. A warning is logged for the remainder.

Output Formats

In non-interactive mode (-N), towl supports five output formats. Terminal-based formats write to stdout; file-based formats require the -o flag with a matching file extension.

Note: Output format flags only apply in non-interactive mode. The interactive TUI has its own display. Use towl scan -N -f <format> to select a format.

Terminal / Table (default)

towl scan -N
# or explicitly:
towl scan -N -f table
towl scan -N -f terminal

Renders an ASCII table to stdout:

┌──────┬─────────────────────────┬──────────────────┬──────┬──────────┐
│ Type │ Description             │ File             │ Line │ Function │
├──────┼─────────────────────────┼──────────────────┼──────┼──────────┤
│ TODO │ Implement caching       │ src/lib/cache.rs │   42 │ process  │
│ FIXME│ Handle timeout          │ src/lib/net.rs   │  108 │ connect  │
└──────┴─────────────────────────┴──────────────────┴──────┴──────────┘

Note: table and terminal are aliases -- both produce the same output.

JSON

towl scan -N -f json -o todos.json

Produces structured JSON with a summary and TODOs grouped by type:

{
  "summary": {
    "total": 2,
    "by_type": {
      "TODO": 1,
      "FIXME": 1
    }
  },
  "todos": {
    "TODO": [
      {
        "id": "abc123",
        "file_path": "src/lib/cache.rs",
        "line_number": 42,
        "column_start": 5,
        "column_end": 30,
        "todo_type": "Todo",
        "description": "Implement caching",
        "original_text": "// TODO: Implement caching",
        "context_lines": ["fn process() {", "    // TODO: Implement caching", "    unimplemented!()"],
        "function_context": "process"
      }
    ]
  }
}

CSV

towl scan -N -f csv -o todos.csv

Produces a CSV file with a header row:

Type,Description,File,Line,Column Start,Column End,Function,Original Text,Context Lines
TODO,Implement caching,src/lib/cache.rs,42,5,30,process,// TODO: Implement caching,"fn process() {|    // TODO: Implement caching|    unimplemented!()"

Context lines are joined with | separators within a single quoted field.

Markdown

towl scan -N -f markdown -o todos.md

Produces a Markdown document with sections grouped by TODO type:

# TODOs

## TODO (1)

### Implement caching
- **File:** src/lib/cache.rs
- **Line:** 42
- **Function:** process

**Context:**
> fn process() {
>     // TODO: Implement caching
>     unimplemented!()

TOML

towl scan -N -f toml -o todos.toml

Produces a TOML file with a summary table and grouped items:

[summary]
total = 2

[summary.by_type]
TODO = 1
FIXME = 1

[[todos.TODO]]
description = "Implement caching"
file_path = "src/lib/cache.rs"
line_number = 42
function_context = "process"

Extension Validation

File-based formats require the output path to have a matching extension:

Mismatched extensions produce an error:

Error: Invalid output path: expected .json extension for JSON format

Choosing a Format

Filtering

towl supports filtering scan results by TODO type using the -t / --todo-type flag.

Filter by Type

towl scan -t todo      # Only TODO comments
towl scan -t fixme     # Only FIXME comments
towl scan -t hack      # Only HACK comments
towl scan -t note      # Only NOTE comments
towl scan -t bug       # Only BUG comments

The filter value is case-insensitive -- TODO, todo, and Todo all work.

Available Types

towl recognises five built-in TODO types:

Each type is matched via the corresponding regex pattern in the todo_patterns configuration. The default patterns are case-insensitive ((?i)).

Combining with Output Formats

Filtering works with any output format:

# FIXMEs as JSON
towl scan -t fixme -f json -o fixmes.json

# BUGs as Markdown
towl scan -t bug -f markdown -o bugs.md

# NOTEs in terminal table
towl scan -t note

Without Filtering

When no -t flag is provided, all recognised types are included in the output. The results are grouped by type in all formats.

Custom Patterns

You can add custom TODO patterns in .towl.toml. Each pattern must contain a capture group (.*) for extracting the description:

[parsing]
todo_patterns = [
    "(?i)\\bTODO:\\s*(.*)",
    "(?i)\\bFIXME:\\s*(.*)",
    "(?i)\\bHACK:\\s*(.*)",
    "(?i)\\bNOTE:\\s*(.*)",
    "(?i)\\bBUG:\\s*(.*)",
    "(?i)\\bXXX:\\s*(.*)",
]

Note: Custom patterns extend the set of matched comments but do not add new filter types to -t. The built-in five types are always available for filtering.

API Overview

towl is structured as a library (towl crate) with a thin binary wrapper. The library exposes modules for configuration, scanning, parsing, output, and error handling.

Module Map

towl (lib)
├── cli          Command-line argument parsing (clap)
├── comment      TODO types and comment structures
│   ├── todo     TodoType enum, TodoComment struct
│   └── error    TowlCommentError
├── config       Configuration loading and validation
│   ├── types    TowlConfig, ParsingConfig, GitHubConfig, Owner, Repo
│   ├── git      GitRepoInfo (git remote discovery)
│   └── error    TowlConfigError
├── scanner      Directory walking and file filtering
│   ├── types    Scanner, ScanResult
│   └── error    TowlScannerError
├── parser       Regex-based TODO extraction
│   ├── types    Parser, Pattern
│   └── error    TowlParserError
├── output       Formatting and writing results
│   ├── formatter
│   │   ├── formatters   CsvFormatter, JsonFormatter, MarkdownFormatter,
│   │   │                TableFormatter, TomlFormatter
│   │   └── error        FormatterError
│   ├── writer
│   │   ├── writers      StdoutWriter, FileWriter
│   │   └── error        WriterError
│   └── error            TowlOutputError
├── github       GitHub issue creation
│   ├── client   GitHubClient
│   ├── types    CreatedIssue
│   └── error    TowlGitHubError
├── processor    TODO replacement with issue links
│   ├── types    Processor, ProcessorResult
│   └── error    TowlProcessorError
├── llm          LLM-powered TODO validation
│   ├── analyse  analyse_todos, gather_expanded_context
│   ├── claude   ClaudeProvider (Anthropic API)
│   ├── openai   OpenAiProvider (OpenAI-compatible API)
│   ├── cli      ClaudeCodeProvider, CodexProvider (CLI agents)
│   ├── prompt   System prompt and user content construction
│   ├── types    AnalysisResult, AnalysisSummary, Validity, LlmUsage
│   └── error    TowlLlmError
├── tui          Interactive terminal UI
│   ├── app      App, AppMode, SortField, PeekState
│   ├── input    Action, handle_input
│   ├── render   draw
│   └── error    TowlTuiError
└── error        Top-level TowlError (aggregates all error types)

Data Flow

TowlConfig ──► Scanner ──► Parser ──► Output
   │              │            │          │
   │              │            │          ├─ FormatterImpl (enum dispatch)
   │              │            │          └─ WriterImpl (enum dispatch)
   │              │            │
   │              │            └─ Vec<TodoComment>
   │              │
   │              └─ ScanResult { todos, files_scanned, ... }
   │
   ├─ ParsingConfig + GitHubConfig + LlmConfig
   │
   └─ LlmConfig ──► LlmProvider ──► analyse_todos ──► AnalysisSummary

Key Types

Error Hierarchy

TowlError
├── TowlConfigError      Config loading, TOML parsing, git discovery
├── TowlScannerError     File walk, I/O, resource limits
│   └── TowlParserError  Regex compilation, pattern validation
├── TowlOutputError      Formatting, file writing
│   ├── FormatterError   Serialisation failures
│   └── WriterError      I/O, path traversal
├── TowlGitHubError      API errors, auth, rate limiting
├── TowlProcessorError   File replacement errors
├── TowlTuiError         Terminal I/O errors
└── TowlLlmError         LLM API, auth, parsing, I/O

All error types use thiserror for Display and Error trait implementations. Conversion between levels uses #[from] attributes for ergonomic ? propagation.

Constants

Scanner

The scanner walks a directory tree, filters files by extension and exclude patterns, reads content, and delegates to the parser for TODO extraction.

Scanner

#![allow(unused)]
fn main() {
pub struct Scanner {
    parser: Parser,
    config: ParsingConfig,
}
}

Constructor

#![allow(unused)]
fn main() {
pub fn new(config: ParsingConfig) -> Result<Self, TowlScannerError>
}

Creates a new scanner. Compiles all regex patterns from the config during construction so pattern errors are caught early.

scan

#![allow(unused)]
fn main() {
pub async fn scan(&self, path: PathBuf) -> Result<ScanResult, TowlScannerError>
}

Recursively scans path for TODO comments. Returns a ScanResult on success.

Behaviour:

1. Validates the path (rejects path traversal)
2. Walks the directory using the ignore crate (respects .gitignore)
3. Filters files by extension (file_extensions config)
4. Skips files matching exclude_patterns
5. Skips files larger than MAX_FILE_SIZE (10 MB)
6. Reads and parses each file asynchronously via tokio::fs
7. Collects results until a resource limit is reached or the walk completes

Errors:

• InvalidPath -- Path contains traversal components (..)
• FileTooLarge -- File exceeds 10 MB
• TooManyTodos -- Single file exceeds 10,000 TODOs
• TooManyFiles -- Walk exceeds 100,000 files
• UnableToReadFileAtPath -- I/O error reading a specific file
• UnableToWalkFile -- Directory walk error
• ParsingError -- Regex or parsing failure (propagated from parser)

ScanResult

#![allow(unused)]
fn main() {
pub struct ScanResult {
    pub todos: Vec<TodoComment>,
    pub files_scanned: usize,
    pub files_skipped: usize,
    pub files_errored: usize,
    pub duration: std::time::Duration,
}
}

Methods

#![allow(unused)]
fn main() {
pub const fn all_files_failed(&self) -> bool
}

Returns true when files_scanned == 0 and files_errored > 0. Indicates a likely permissions or path issue where no files could be read.

#![allow(unused)]
fn main() {
pub const fn is_clean(&self) -> bool
}

Returns true when todos is empty and files_errored == 0. A clean scan with no issues.

Resource Limits

Example

#![allow(unused)]
fn main() {
use towl::config::ParsingConfig;
use towl::scanner::Scanner;
use std::path::PathBuf;

let config = ParsingConfig::default();
let scanner = Scanner::new(config)?;
let result = scanner.scan(PathBuf::from(".")).await?;

println!("Found {} TODOs in {} files", result.todos.len(), result.files_scanned);

if result.all_files_failed() {
    eprintln!("Warning: no files could be read");
}
}

Parser

The parser reads file content, identifies comment lines using regex patterns, extracts TODO items, and captures surrounding context.

Parser

#![allow(unused)]
fn main() {
pub struct Parser {
    comment_patterns: Vec<Regex>,
    patterns: Vec<Pattern>,
    function_patterns: Vec<Regex>,
    context_lines: usize,
}
}

The parser is pub(crate) -- it is used internally by Scanner and not exposed in the public API. The public interface is through the module-level functions.

Construction

Created internally by Scanner::new() using Parser::new(config). All regex patterns are compiled once during construction.

Public Functions

validate_patterns

#![allow(unused)]
fn main() {
pub fn validate_patterns(config: &ParsingConfig) -> Result<(), TowlParserError>
}

Validates all regex patterns in the config without creating a parser. Useful for checking configuration before starting a scan.

Checks:

• Each pattern is valid regex
• Each pattern is within MAX_PATTERN_LENGTH (256 characters)
• Compiled regex is within REGEX_SIZE_LIMIT (256 KB)

parse_content

#![allow(unused)]
fn main() {
pub fn parse_content(
    config: &ParsingConfig,
    path: &Path,
    content: &str,
) -> Result<Vec<TodoComment>, TowlParserError>
}

Parses file content for TODO comments. Creates a temporary parser, runs extraction, and returns the results.

Parsing Pipeline

For each line in the file:

1. Comment detection -- Check if the line matches any comment_prefixes pattern
2. TODO matching -- Check if the comment matches any todo_patterns pattern
3. Type classification -- Determine the TodoType from the matched pattern
4. Description extraction -- Extract the description via the first capture group (.*)
5. Context capture -- Grab include_context_lines lines above and below
6. Function detection -- Search upward (within 3 lines) for a function_patterns match

Pattern Types

Comment Prefixes

Regex patterns that identify comment lines:

TODO Patterns

Regex patterns with a capture group for the description:

All default patterns are case-insensitive ((?i)).

Function Patterns

Regex patterns to detect enclosing function names:

Constants

Errors

#![allow(unused)]
fn main() {
pub enum TowlParserError {
    InvalidRegexPattern(String, regex::Error),
    UnknownConfigPattern(TowlCommentError),
    RegexGroupMissing,
    PatternTooLong(usize, usize),
    TooManyTotalPatterns { count: usize, max_allowed: usize },
}
}

Config

The config module loads settings from .towl.toml, merges environment variables, and provides the init command for generating default configuration.

TowlConfig

#![allow(unused)]
fn main() {
pub struct TowlConfig {
    pub parsing: ParsingConfig,
    pub github: GitHubConfig,
    pub llm: LlmConfig,
}
}

TowlConfig::load

#![allow(unused)]
fn main() {
impl TowlConfig {
    pub fn load(path: Option<&PathBuf>) -> Result<Self, TowlConfigError>;
}
}

Loads configuration with this precedence:

1. Built-in defaults
2. Config file resolved as: explicit path argument > TOWL_CONFIG env var > .towl.toml
3. Git remote auto-detection for owner/repo
4. Environment variable overrides (TOWL_GITHUB_*, TOWL_LLM_*)

If no config file exists, defaults are used without error.

init

#![allow(unused)]
fn main() {
pub async fn init(path: &Path, force: bool) -> Result<(), TowlConfigError>
}

Creates a .towl.toml file at the given path. Validates that a GitHub git remote exists but does not write owner/repo to the file (they are always detected at runtime).

• Fails if the file already exists (unless force is true)
• Validates the path for traversal attacks
• Serializes ParsingConfig and LlmConfig defaults to TOML

ParsingConfig

#![allow(unused)]
fn main() {
pub struct ParsingConfig {
    pub file_extensions: HashSet<String>,
    pub exclude_patterns: Vec<String>,
    pub include_context_lines: usize,
    pub comment_prefixes: Vec<String>,
    pub todo_patterns: Vec<String>,
    pub function_patterns: Vec<String>,
}
}

All fields have defaults via #[serde(default)]:

Each pattern array is limited to MAX_CONFIG_PATTERNS (100) entries.

GitHubConfig

#![allow(unused)]
fn main() {
pub struct GitHubConfig {
    pub token: SecretString,
    pub owner: Owner,
    pub repo: Repo,
    pub rate_limit_delay_ms: u64,
}
}

• token is stored as secrecy::SecretString and masked in debug/display output
• owner and repo are auto-detected from git remote get-url origin at runtime (not serialised to config)
• rate_limit_delay_ms adds a delay between GitHub API calls (default: 1000ms)

Environment Variable Overrides

Owner / Repo

Validated newtype wrappers providing type safety:

#![allow(unused)]
fn main() {
pub struct Owner(String);
pub struct Repo(String);
}

try_new

#![allow(unused)]
fn main() {
pub fn try_new(s: impl Into<String>) -> Result<Self, TowlConfigError>
}

Constructs a new Owner or Repo, rejecting values exceeding MAX_CONFIG_STRING_LENGTH (512 characters).

Errors:

• ConfigValueTooLong -- Value exceeds 512 characters

Both also implement:

• Display, Default, Debug, Clone, PartialEq, Eq
• Serialize, Deserialize

LlmConfig

#![allow(unused)]
fn main() {
pub struct LlmConfig {
    pub provider: String,
    pub model: String,
    pub base_url: Option<String>,
    pub api_key: SecretString,
    pub max_concurrent_analyses: usize,
    pub max_analyse_count: usize,
    pub max_tokens: u32,
    pub max_retries: usize,
    pub command: Option<String>,
    pub args: Option<Vec<String>>,
}
}

• api_key is stored as secrecy::SecretString and masked in debug output (env-only via TOWL_LLM_API_KEY)
• provider selects the LLM backend: "claude", "openai", "claude-code", or "codex"
• command and args allow overriding the CLI binary path and arguments for CLI providers

Environment Variable Overrides

GitRepoInfo (internal)

#![allow(unused)]
fn main() {
pub(crate) struct GitRepoInfo {
    pub owner: Owner,
    pub repo: Repo,
}
}

from_path

#![allow(unused)]
fn main() {
pub(crate) async fn from_path<P: AsRef<Path>>(path: P) -> Result<Self, TowlConfigError>
}

Internal function that discovers the git remote URL by running git remote get-url origin and parses the owner and repo name. Supports both HTTPS and SSH URL formats. Not part of the public API.

Errors:

• GitRepoNotFound -- Not inside a git repository
• GitRemoteNotFound -- No origin remote configured
• GitInvalidUrl -- Could not parse owner/repo from the URL

Errors

#![allow(unused)]
fn main() {
pub enum TowlConfigError {
    PathTraversalAttempt(PathBuf),
    ConfigAlreadyExists(PathBuf),
    WriteToFileError(PathBuf, std::io::Error),
    UnableToParseToml(toml::ser::Error),
    CouldNotCreateConfig(ConfigError),
    GitRepoNotFound { message: String },
    GitRemoteNotFound { message: String },
    GitInvalidUrl { url: String, message: String },
    TooManyConfigPatterns { field: String, count: usize, max_allowed: usize },
    ConfigValueTooLong { field: String, length: usize, max_length: usize },
    ContextLinesOutOfRange { value: usize, min: usize, max: usize },
}
}

Constants

Output

The output module combines a formatter and a writer to produce scan results in the requested format and destination.

Output

#![allow(unused)]
fn main() {
pub struct Output {
    writer: WriterImpl,
    formatter: FormatterImpl,
}
}

Constructor

#![allow(unused)]
fn main() {
pub fn new(
    output_format: OutputFormat,
    output_path: Option<PathBuf>,
) -> Result<Self, TowlOutputError>
}

Creates an output handler by selecting the appropriate formatter and writer.

Format-to-writer mapping:

save

#![allow(unused)]
fn main() {
pub async fn save(&self, todos: &[TodoComment]) -> Result<(), TowlOutputError>
}

Formats the TODOs and writes them to the destination. TODOs are grouped by type before formatting.

OutputFormat

#![allow(unused)]
fn main() {
pub enum OutputFormat {
    Table,
    Json,
    Csv,
    Toml,
    Markdown,
    Terminal,
}
}

Used as a CLI argument via clap::ValueEnum. Table and Terminal are treated identically.

Formatter Dispatch

Internally, FormatterImpl is an enum that dispatches to the correct formatter without dynamic dispatch:

#![allow(unused)]
fn main() {
pub(crate) enum FormatterImpl {
    Csv(CsvFormatter),
    Json(JsonFormatter),
    Markdown(MarkdownFormatter),
    Table(TableFormatter),
    Toml(TomlFormatter),
}
}

Each formatter implements the internal Formatter trait:

#![allow(unused)]
fn main() {
pub(crate) trait Formatter {
    fn format(
        &self,
        todos: &HashMap<&TodoType, Vec<&TodoComment>>,
        total_count: usize,
    ) -> Result<Vec<String>, FormatterError>;
}
}

Writer Dispatch

WriterImpl dispatches between stdout and file output:

#![allow(unused)]
fn main() {
pub(crate) enum WriterImpl {
    Stdout(StdoutWriter),
    File(FileWriter),
}
}

Each writer implements the internal Writer trait:

#![allow(unused)]
fn main() {
pub(crate) trait Writer {
    async fn write(&self, content: Vec<String>) -> Result<(), WriterError>;
}
}

FileWriter

Validates the output path on construction:

• Rejects path traversal (.. components)
• Resolves symlinks before writing

StdoutWriter

Writes each formatted line to stdout followed by a newline.

Errors

TowlOutputError

#![allow(unused)]
fn main() {
pub enum TowlOutputError {
    InvalidOutputPath(String),
    UnableToFormatTodos(FormatterError),
    UnableToWriteTodos(WriterError),
}
}

FormatterError

#![allow(unused)]
fn main() {
pub enum FormatterError {
    SerializationError(String),
    IntegerOverflow(usize),
}
}

WriterError

#![allow(unused)]
fn main() {
pub enum WriterError {
    IoError(std::io::Error),
    PathTraversal(PathBuf),
}
}

GitHub

The GitHub module creates issues from TODO comments, detects duplicates, and handles rate limiting.

GitHubClient

#![allow(unused)]
fn main() {
pub struct GitHubClient {
    // private fields
}
}

Authenticated GitHub API client for creating issues from TODO comments. Maintains a cache of existing issue titles and TODO IDs for deduplication. Includes rate-limit handling with configurable delays and automatic retries.

new

#![allow(unused)]
fn main() {
pub fn new(config: &GitHubConfig) -> Result<Self, TowlGitHubError>
}

Creates a new client from a GitHubConfig. Exposes the SecretString token once to build the Octocrab API client.

Errors:

• MissingToken -- Token is empty
• ApiError -- Octocrab client failed to build

load_existing_issues

#![allow(unused)]
fn main() {
pub async fn load_existing_issues(&mut self) -> Result<(), TowlGitHubError>
}

Paginates through all existing issues (open and closed) in the repository, caching their titles and embedded TODO IDs. Call this before create_issue to enable duplicate detection.

Errors:

• ApiError -- GitHub API call failed
• AuthError -- Invalid or expired token
• RepositoryNotFound -- Owner/repo combination does not exist

issue_exists

#![allow(unused)]
fn main() {
pub fn issue_exists(&self, todo: &TodoComment) -> bool
}

Returns true if a matching issue already exists, checked by TODO ID (embedded in issue body) or by generated title.

create_issue

#![allow(unused)]
fn main() {
pub async fn create_issue(
    &mut self,
    todo: &TodoComment,
) -> Result<CreatedIssue, TowlGitHubError>
}

Creates a GitHub issue for a TODO comment. Generates a title with type prefix, truncated description, and file location. The body includes file path, line number, column range, description, function context, original comment, and surrounding code.

Automatically retries on rate limiting (up to 3 attempts).

Errors:

• IssueAlreadyExists -- Duplicate detected
• RateLimitExceeded -- Rate limit hit after max retries
• ApiError -- GitHub API failure
• AuthError -- Authentication failure

Issue Title Format

[TODO] Implement caching (cache.rs:42)

Titles are capped at 50 characters (excluding the type prefix and location suffix). Long descriptions are truncated at word boundaries with ....

Issue Body Sections

1. TODO Details -- Type, file, line, column range
2. Description -- Extracted description text (Markdown-escaped)
3. Function Context -- Enclosing function name (if detected)
4. Original Comment -- Full comment line in a code block
5. Context -- Surrounding source lines in a code block
6. TODO ID -- Embedded identifier for deduplication

Duplicate Detection

Issues are deduplicated by two methods:

1. TODO ID -- Each issue body contains *TODO ID: {file_path}_L{line_number}*. If any existing issue body contains the same ID, the TODO is skipped.
2. Title match -- If the generated title matches an existing issue title exactly, the TODO is skipped.

CreatedIssue

#![allow(unused)]
fn main() {
pub struct CreatedIssue {
    pub number: u64,
    pub title: String,
    pub html_url: String,
    pub todo_id: String,
}
}

Metadata for a successfully created GitHub issue. Implements Serialize and Deserialize for JSON roundtripping.

Errors

#![allow(unused)]
fn main() {
pub enum TowlGitHubError {
    ApiError { message: String, source: Option<octocrab::Error> },
    AuthError,
    RateLimitExceeded { retry_after_secs: u64 },
    IssueAlreadyExists { title: String },
    RepositoryNotFound { owner: String, repo: String },
    MissingToken,
}
}

Example

#![allow(unused)]
fn main() {
use towl::config::TowlConfig;
use towl::github::GitHubClient;

let config = TowlConfig::load(None)?;
let mut client = GitHubClient::new(&config.github)?;

// Load existing issues for duplicate detection
client.load_existing_issues().await?;

// Create an issue (skips if duplicate)
if !client.issue_exists(&todo) {
    let issue = client.create_issue(&todo).await?;
    println!("Created #{}: {}", issue.number, issue.html_url);
}
}

LLM

The LLM module provides AI-powered TODO validation using Claude (Anthropic API) or any OpenAI-compatible endpoint.

LlmProvider

#![allow(unused)]
fn main() {
pub enum LlmProvider {
    Claude(ClaudeProvider),
    OpenAi(OpenAiProvider),
    ClaudeCode(ClaudeCodeProvider),
    Codex(CodexProvider),
}
}

Dispatches LLM calls to the configured provider. Follows towl's existing enum dispatch pattern (FormatterImpl, WriterImpl).

call_raw

#![allow(unused)]
fn main() {
pub async fn call_raw(
    &self,
    user_content: &str,
    system_prompt: &str,
    api_key: &SecretString,
) -> Result<(String, LlmUsage), TowlLlmError>
}

Sends a prompt to the LLM and returns the response text and token usage.

build_provider

#![allow(unused)]
fn main() {
pub fn build_provider(config: &LlmConfig) -> Result<LlmProvider, TowlLlmError>
}

Factory function that creates the appropriate provider from configuration.

analyse_todos

#![allow(unused)]
fn main() {
pub async fn analyse_todos(
    todos: &mut [TodoComment],
    config: &LlmConfig,
    on_progress: impl FnMut(usize, usize),
) -> Result<AnalysisSummary, TowlLlmError>
}

Main entry point for TODO analysis. Calls on_progress(completed, total) after each TODO is analysed, allowing callers to render progress feedback (e.g. a progress bar).

For each TODO (up to max_analyse_count):

1. Reads expanded context (~30 lines around the TODO + full function body)
2. Constructs a prompt with the TODO description, file path, and code context
3. Calls the LLM to determine validity
4. Parses the structured JSON response into an AnalysisResult
5. Attaches the result to TodoComment.analysis
6. Calls on_progress with the current count

Errors:

• NotConfigured -- TOWL_LLM_API_KEY not set
• UnsupportedProvider -- Provider is not "claude" or "openai"
• ApiError, AuthError, RateLimited -- From the LLM API

Validity

#![allow(unused)]
fn main() {
pub enum Validity {
    Valid,
    Invalid,
    Uncertain,
}
}

Whether a TODO is still valid:

AnalysisResult

#![allow(unused)]
fn main() {
pub struct AnalysisResult {
    pub validity: Validity,
    pub reasoning: String,
    pub is_resolved: bool,
    pub is_relevant: bool,
    pub is_actionable: bool,
    pub confidence: f64,
    pub enrichment: String,
}
}

AnalysisSummary

#![allow(unused)]
fn main() {
pub struct AnalysisSummary {
    pub valid_count: usize,
    pub invalid_count: usize,
    pub uncertain_count: usize,
    pub error_count: usize,
}
}

Summary counts returned by analyse_todos().

Providers

ClaudeProvider

POST to https://api.anthropic.com/v1/messages with headers:

• x-api-key: API key
• anthropic-version: 2023-06-01

System prompt is a top-level system field (not in the messages array).

OpenAiProvider

POST to {base_url}/chat/completions with Authorization: Bearer {key}. Default base URL: https://api.openai.com/v1. Configurable for Ollama, vLLM, etc.

System prompt is the first message in the messages array with role: "system".

ClaudeCodeProvider

Invokes the claude CLI as a subprocess with -p --output-format json. The combined system prompt and user content are passed as the final argument. No API key required.

Default command: claude. Configurable via llm.command and llm.args.

Auto-falls back to ClaudeProvider (API) if the CLI binary is not found on PATH.

CodexProvider

Invokes the codex CLI as a subprocess with -q. The combined prompt is passed as the final argument. No API key required.

Default command: codex. Configurable via llm.command and llm.args.

Auto-falls back to OpenAiProvider (API) with gpt-4o if the CLI binary is not found on PATH.

is_cli_provider

#![allow(unused)]
fn main() {
pub const fn is_cli_provider(&self) -> bool
}

Returns true for ClaudeCode and Codex variants. Used to skip the API key requirement for CLI-based providers.

Configuration

See Configuration for the [llm] config section.

Errors

#![allow(unused)]
fn main() {
pub enum TowlLlmError {
    ApiError { message: String, status: Option<u16> },
    AuthError,
    RateLimited { retry_after_secs: u64 },
    ParseError { message: String },
    NotConfigured,
    UnsupportedProvider { provider: String },
    IoError { message: String },
}
}

Retryable Errors

TowlLlmError implements is_retryable() which returns true for:

• RateLimited -- always retryable
• ApiError with status >= 500 -- server errors
• ApiError with no status -- network failures

Processor

The processor replaces TODO comments in source files with GitHub issue links after issues are created.

Processor

#![allow(unused)]
fn main() {
pub struct Processor;
}

Stateless processor that operates on batches of (TodoComment, CreatedIssue) pairs. All methods are associated functions (no self).

replace_todos

#![allow(unused)]
fn main() {
pub async fn replace_todos(
    repo_root: &Path,
    replacements: &[(TodoComment, CreatedIssue)],
) -> ProcessorResult
}

Replaces TODO comments in source files with GH_ISSUE: <issue_url> links.

Behaviour:

1. Groups replacements by file path for efficient batch processing
2. For each file, validates the path stays within repo_root
3. Reads file content, replaces each TODO line, writes back atomically
4. Returns a ProcessorResult with counts and per-file errors

Path safety:

• Both the file path and repo root are canonicalised before comparison
• Files outside the repo root are rejected with PathOutsideRoot
• Issue URLs must start with https://github.com/ or are rejected

Replacement format:

The comment prefix (e.g., // , # , /* ) is preserved. The TODO text after the prefix is replaced:

// TODO: Implement caching    -->    // GH_ISSUE: https://github.com/owner/repo/issues/42
# FIXME: Handle timeout        -->    # GH_ISSUE: https://github.com/owner/repo/issues/43

Atomic writes:

Files are written via a tempfile in the same directory, then atomically persisted. This prevents partial writes if the process is interrupted.

Empty input:

If replacements is empty, returns immediately with zero counts and no I/O.

ProcessorResult

#![allow(unused)]
fn main() {
pub struct ProcessorResult {
    pub files_modified: usize,
    pub todos_replaced: usize,
    pub errors: Vec<(PathBuf, TowlProcessorError)>,
}
}

Summary of a batch replacement operation. The errors field contains per-file errors that did not abort the overall operation -- other files continue processing.

Errors

#![allow(unused)]
fn main() {
pub enum TowlProcessorError {
    FileReadError(PathBuf, std::io::Error),
    FileWriteError(PathBuf, std::io::Error),
    LineOutOfBounds { path: PathBuf, line: usize, total_lines: usize },
    CommentPrefixNotFound { path: PathBuf, line: usize },
    PathOutsideRoot { path: PathBuf, root: PathBuf },
    InvalidIssueUrl { url: String },
}
}

Example

#![allow(unused)]
fn main() {
use towl::processor::Processor;

let replacements = vec![(todo, created_issue)];
let result = Processor::replace_todos(repo_root, &replacements).await;

println!("Modified {} files, replaced {} TODOs", result.files_modified, result.todos_replaced);

for (path, err) in &result.errors {
    eprintln!("Error in {}: {}", path.display(), err);
}
}

TUI

The TUI module provides an interactive terminal interface for browsing, filtering, and acting on TODO comments. Built on ratatui and crossterm.

run

#![allow(unused)]
fn main() {
pub fn run(
    todos: Vec<TodoComment>,
    github_config: &GitHubConfig,
    repo_root: &Path,
) -> Result<(), TowlTuiError>
}

Launches the interactive TUI. Takes ownership of the terminal (raw mode, alternate screen). Terminal state is always restored on exit, even on error.

Errors:

• TowlTuiError::Io -- Terminal I/O failure

App

#![allow(unused)]
fn main() {
pub struct App {
    // private fields
}
}

Core TUI application state. Manages the TODO list, selection set, cursor position, filtering, sorting, and mode transitions.

Constructor

#![allow(unused)]
fn main() {
pub fn new(todos: Vec<TodoComment>) -> Self
}

Creates a new app with all TODOs visible, no selection, sorted by file path.

State Accessors

Navigation

Selection

Filtering and Sorting

Mode Transitions

AppMode

#![allow(unused)]
fn main() {
pub enum AppMode {
    Browse,
    Peek(PeekState),
    Confirm,
    Creating(CreatingState),
    Done(DoneState),
    DeleteConfirm(Vec<TodoComment>),
}
}

The current UI mode determines which view is rendered and which keys are active.

SortField

#![allow(unused)]
fn main() {
pub enum SortField {
    File,
    Line,
    Priority,
    Type,
}
}

Field used to sort the TODO list. Cycle with the s key in Browse mode.

• File -- Sort by file path, then by line number within each file
• Line -- Sort by line number globally
• Priority -- Sort by TODO type priority (Bug=1, Fixme=2, Hack=3, Todo=4, Note=5)
• Type -- Sort alphabetically by type name

Supporting Types

PeekState

#![allow(unused)]
fn main() {
pub struct PeekState {
    pub lines: Vec<(usize, String)>,
    pub file: String,
    pub todo_line: usize,
    pub scroll: usize,
    pub analysis: Option<AnalysisResult>,
}
}

State for the source-code peek overlay. Contains numbered source lines around the TODO, with scroll position. When --ai is active, analysis holds the LLM result -- the reasoning text is word-wrapped to the popup width during rendering.

CreatingState

#![allow(unused)]
fn main() {
pub struct CreatingState {
    pub phase: String,
    pub progress: usize,
    pub total: usize,
    pub errors: Vec<String>,
    pub created_issues: Vec<CreatedIssue>,
}
}

State tracked during background GitHub issue creation. Updated via channel messages from the spawned task.

DoneState

#![allow(unused)]
fn main() {
pub struct DoneState {
    pub created_issues: Vec<CreatedIssue>,
    pub errors: Vec<String>,
}
}

Final state after issue creation completes, showing results and any errors.

Action

#![allow(unused)]
fn main() {
pub enum Action {
    Continue,
    Quit,
}
}

Result of processing a keyboard event. Continue keeps the event loop running; Quit exits the TUI.

handle_input

#![allow(unused)]
fn main() {
pub fn handle_input(
    app: &mut App,
    timeout: std::time::Duration,
) -> std::io::Result<Action>
}

Polls for keyboard input and dispatches to mode-specific handlers. Returns Action::Quit on q, Esc (in appropriate modes), or Ctrl+C.

Errors

#![allow(unused)]
fn main() {
pub enum TowlTuiError {
    Io(std::io::Error),
}
}

Terminal I/O errors from crossterm or ratatui operations.

Types

Core data types used across the towl library.

TodoType

#![allow(unused)]
fn main() {
pub enum TodoType {
    Todo,
    Fixme,
    Hack,
    Note,
    Bug,
}
}

Represents the category of a TODO comment.

Display

as_filter_str

#![allow(unused)]
fn main() {
pub const fn as_filter_str(&self) -> &'static str
}

Returns the lowercase filter string used for CLI filtering:

Conversions

• TryFrom<&str> -- Case-insensitive conversion from string
• clap::ValueEnum -- CLI argument parsing

Trait Implementations

Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, ValueEnum

TodoComment

#![allow(unused)]
fn main() {
pub struct TodoComment {
    pub id: String,
    pub file_path: PathBuf,
    pub line_number: usize,
    pub column_start: usize,
    pub column_end: usize,
    pub todo_type: TodoType,
    pub original_text: String,
    pub description: String,
    pub context_lines: Vec<String>,
    pub function_context: Option<String>,
    pub analysis: Option<AnalysisResult>,
}
}

A single TODO comment extracted from a source file.

Trait Implementations

Debug, Clone, PartialEq, Serialize, Deserialize

ScanResult

#![allow(unused)]
fn main() {
pub struct ScanResult {
    pub todos: Vec<TodoComment>,
    pub files_scanned: usize,
    pub files_skipped: usize,
    pub files_errored: usize,
    pub duration: std::time::Duration,
}
}

Returned by Scanner::scan(). See Scanner for details.

Owner / Repo

Newtype wrappers for GitHub owner and repository names:

#![allow(unused)]
fn main() {
pub struct Owner(String);
pub struct Repo(String);
}

Both provide try_new(impl Into<String>) -> Result<Self, TowlConfigError> (validates length) and Display. See Config for details.

OutputFormat

#![allow(unused)]
fn main() {
pub enum OutputFormat {
    Table,
    Json,
    Csv,
    Toml,
    Markdown,
    Terminal,
}
}

CLI-facing enum for selecting output format. Table and Terminal produce identical output. See Output for details.

Errors

towl uses typed errors throughout, built with thiserror. Each module defines its own error enum, and the top-level TowlError aggregates them.

Error Hierarchy

TowlError
├── TowlConfigError
├── TowlScannerError
│   └── TowlParserError
│       └── TowlCommentError
├── TowlOutputError
│   ├── FormatterError
│   └── WriterError
├── TowlGitHubError
├── TowlProcessorError
├── TowlTuiError
└── TowlLlmError

TowlError

Top-level error type used by the CLI binary. All sub-error types convert automatically via #[from].

#![allow(unused)]
fn main() {
pub enum TowlError {
    Config(TowlConfigError),
    Scanner(TowlScannerError),
    Output(TowlOutputError),
    GitHub(TowlGitHubError),
    Processor(TowlProcessorError),
    Tui(TowlTuiError),
    Llm(TowlLlmError),
}
}

TowlConfigError

Errors during configuration loading, initialisation, and validation.

TowlScannerError

Errors during directory walking and file reading.

TowlParserError

Errors during regex compilation and TODO extraction.

TowlCommentError

Errors in comment type resolution.

TowlOutputError

Errors during formatting and writing.

FormatterError

Errors in output formatting.

WriterError

Errors in output writing.

TowlGitHubError

Errors from GitHub API interactions.

TowlProcessorError

Errors from replacing TODO comments with issue links in source files.

TowlLlmError

Errors from LLM API interactions and analysis.

is_retryable() returns true for RateLimited, ApiError with status >= 500, and ApiError with no status (network failures).

TowlTuiError

Errors from the interactive terminal UI.

Error Propagation

Errors propagate upward using ? and #[from]:

TowlCommentError  -->  TowlParserError    -->  TowlScannerError  -->  TowlError
FormatterError    -->  TowlOutputError     -->  TowlError
WriterError       -->  TowlOutputError     -->  TowlError
TowlGitHubError   ---------------------------->  TowlError
TowlProcessorError --------------------------->  TowlError
TowlTuiError      ---------------------------->  TowlError
TowlLlmError      ---------------------------->  TowlError

All errors implement std::fmt::Display with human-readable messages and std::error::Error for standard error handling.

Architecture

towl follows a pipeline architecture: Config -> Scanner -> Parser -> TUI / Output. Each stage is a separate module with clear boundaries and typed errors.

Pipeline

                ┌──────────┐
                │  Config   │  --config / TOWL_CONFIG / .towl.toml + env vars
                └────┬─────┘
                     │
                ┌────▼─────┐
                │  Scanner  │  Walks directory tree, scans files concurrently
                └────┬─────┘
                     │
                ┌────▼─────┐
                │  Parser   │  Matches comment prefixes + TODO patterns
                └────┬─────┘
                     │
              ┌──────┼──────┐
              │      │      │
        ┌─────▼────┐ │ ┌────▼─────┐
        │   TUI    │ │ │  Output   │  Non-interactive: formats + writes
        │ (default)│ │ │  (-N)     │
        └─────┬────┘ │ └──────────┘
              │      │
        ┌─────▼─────┐│
        │ Processor  ││  Replaces TODOs with GitHub issue links
        └───────────┘│
                ┌────▼─────┐
                │   LLM     │  --ai: validates TODOs with AI
                └──────────┘

Module Boundaries

Config (src/lib/config/)

• Resolves config file path: --config flag > TOWL_CONFIG env var > .towl.toml
• Loads config using the config crate
• Merges environment variable overrides (TOWL_GITHUB_*, TOWL_LLM_*)
• Discovers GitHub owner/repo from git remote get-url origin
• Validates pattern array sizes
• Produces TowlConfig containing ParsingConfig + GitHubConfig + LlmConfig

Submodules:

• types.rs -- TowlConfig, ParsingConfig, GitHubConfig
• defaults.rs -- Default values for config fields
• display.rs -- Display implementation for config tree view
• newtypes.rs -- Owner and Repo newtype wrappers
• validation.rs -- Config validation logic
• git.rs -- GitRepoInfo for parsing git remotes
• error.rs -- TowlConfigError

Scanner (src/lib/scanner/)

• Accepts a ParsingConfig and a root path
• Walks the directory tree using the ignore crate (respects .gitignore)
• Filters files by extension and exclude patterns
• Scans files concurrently with bounded parallelism (up to 64 files)
• Reads files asynchronously via tokio::fs
• Enforces resource limits (file size, TODO counts, file counts)
• Delegates content parsing to the Parser
• Returns ScanResult with TODOs and scan metrics

Submodules:

• types.rs -- Scanner implementation
• limits.rs -- ScanResult and resource limit constants
• walker.rs -- Directory walker construction
• error.rs -- TowlScannerError

Parser (src/lib/parser/)

• Compiles regex patterns once during construction
• Identifies comment lines via comment_prefixes
• Extracts TODO items via todo_patterns
• Captures context lines (configurable window, 1-50)
• Detects enclosing function names via function_patterns
• Produces Vec<TodoComment>

Submodules:

• types.rs -- Parser implementation
• context.rs -- Context line extraction logic
• pattern.rs -- Pattern compilation and matching
• error.rs -- TowlParserError

TUI (src/lib/tui/)

• Full-screen terminal interface using ratatui and crossterm
• Browse, filter, sort, and peek at TODOs
• Select TODOs and create GitHub issues with progress tracking
• Replaces TODO comments in source files with issue links via the Processor

Submodules:

• app.rs -- App state machine and AppMode enum (Browse, Peek, Confirm, Creating, Done)
• input.rs -- Keyboard event handling and action dispatch
• render.rs -- UI rendering (list, peek popup, confirm dialog, progress view)
• error.rs -- TowlTuiError

LLM (src/lib/llm/)

• AI-powered TODO validation using Claude, OpenAI, or local CLI agents
• Enum-dispatched providers following the same pattern as FormatterImpl/WriterImpl
• Gathers expanded context (~30 lines) and full function bodies for each TODO
• Constructs structured prompts and parses JSON responses
• Retry logic with exponential backoff via backon
• CLI providers (claude-code, codex) auto-fall back to API providers if the binary is not on PATH

Submodules:

• analyse.rs -- analyse_todos(), gather_expanded_context(), retry logic
• claude.rs -- ClaudeProvider (Anthropic Messages API)
• openai.rs -- OpenAiProvider (OpenAI Chat Completions API)
• cli.rs -- ClaudeCodeProvider, CodexProvider (subprocess-based)
• prompt.rs -- System prompt and user content construction
• types.rs -- AnalysisResult, AnalysisSummary, Validity, LlmUsage, JSON extraction
• error.rs -- TowlLlmError

Processor (src/lib/processor/)

• Replaces TODO comments in source files with GitHub issue links
• Groups replacements by file for efficient batch processing
• Validates file paths stay within the repository root
• Returns ProcessorResult with counts and error details

Submodules:

• types.rs -- Processor and ProcessorResult
• error.rs -- TowlProcessorError

GitHub (src/lib/github/)

• Creates GitHub issues from TodoComment items via the Octocrab API
• Loads existing issues to detect and skip duplicates
• Constructs issue titles and bodies with file/line metadata

Output (src/lib/output/)

• Combines a FormatterImpl and a WriterImpl
• Groups TODOs by type before formatting
• Uses enum dispatch (not trait objects) for zero-cost abstraction

Output
├── FormatterImpl (enum dispatch)
│   ├── CsvFormatter
│   ├── JsonFormatter
│   ├── MarkdownFormatter
│   ├── TableFormatter
│   └── TomlFormatter
└── WriterImpl (enum dispatch)
    ├── StdoutWriter
    └── FileWriter

Key Design Decisions

Enum Dispatch Over Trait Objects

Both FormatterImpl and WriterImpl use enum variants rather than Box<dyn Trait>. This provides:

• Static dispatch (no vtable overhead)
• Exhaustive matching at compile time
• Simpler lifetime management

Regex Compilation Strategy

All regex patterns are compiled once during Scanner::new() / Parser::new() and reused for every file. This avoids per-file compilation overhead.

Concurrent File Scanning

The scanner discovers all scannable files first, then scans them concurrently using futures::stream::buffer_unordered with a concurrency limit of 64. This provides significant speedup on large codebases while bounding resource usage.

Async I/O

File reading uses tokio::fs for non-blocking I/O. The scanner is async, allowing integration into async applications. The CLI uses #[tokio::main].

TUI Event Loop

The TUI uses a synchronous event loop with crossterm polling. GitHub issue creation runs in a background tokio task, communicating progress back to the UI via an mpsc channel. This keeps the UI responsive during network operations.

Error Type Hierarchy

Each module owns its error type. Errors propagate upward via #[from] conversions:

TowlCommentError → TowlParserError → TowlScannerError → TowlError
FormatterError → TowlOutputError → TowlError
WriterError → TowlOutputError → TowlError
TowlProcessorError → TowlError
TowlTuiError → TowlError
TowlLlmError → TowlError

Newtype Pattern

Owner and Repo are newtype wrappers over String, preventing accidental misuse (e.g., passing an owner where a repo is expected).

Secret Handling

The GitHub token is stored as secrecy::SecretString, which:

• Masks the value in Debug and Display output
• Zeroes memory on drop
• Prevents accidental logging

Directory Layout

src/
├── bin/
│   └── towl.rs              CLI binary
└── lib/
    ├── mod.rs                Library root
    ├── cli/
    │   └── mod.rs            Clap argument definitions
    ├── comment/
    │   ├── mod.rs
    │   ├── todo.rs           TodoType, TodoComment
    │   └── error.rs          TowlCommentError
    ├── config/
    │   ├── mod.rs
    │   ├── types.rs          TowlConfig, ParsingConfig, GitHubConfig
    │   ├── defaults.rs       Default config values
    │   ├── display.rs        Config Display implementation
    │   ├── newtypes.rs       Owner, Repo newtypes
    │   ├── validation.rs     Config validation
    │   ├── git.rs            GitRepoInfo
    │   └── error.rs          TowlConfigError
    ├── scanner/
    │   ├── mod.rs
    │   ├── types.rs          Scanner
    │   ├── limits.rs         ScanResult, resource limits
    │   ├── walker.rs         Directory walker construction
    │   └── error.rs          TowlScannerError
    ├── parser/
    │   ├── mod.rs
    │   ├── types.rs          Parser
    │   ├── context.rs        Context line extraction
    │   ├── pattern.rs        Pattern compilation
    │   └── error.rs          TowlParserError
    ├── github/
    │   ├── mod.rs
    │   ├── client.rs         GitHubClient
    │   ├── types.rs          CreatedIssue
    │   └── error.rs          TowlGitHubError
    ├── llm/
    │   ├── mod.rs             LlmProvider enum dispatch
    │   ├── analyse.rs         analyse_todos, gather_expanded_context
    │   ├── claude.rs          ClaudeProvider
    │   ├── openai.rs          OpenAiProvider
    │   ├── cli.rs             ClaudeCodeProvider, CodexProvider
    │   ├── prompt.rs          System prompt construction
    │   ├── types.rs           AnalysisResult, Validity, JSON extraction
    │   └── error.rs           TowlLlmError
    ├── processor/
    │   ├── mod.rs
    │   ├── types.rs          Processor, ProcessorResult
    │   └── error.rs          TowlProcessorError
    ├── tui/
    │   ├── mod.rs             TUI entry point and event loop
    │   ├── app.rs             App state machine, AppMode
    │   ├── input.rs           Keyboard input handling
    │   ├── render.rs          UI rendering
    │   └── error.rs           TowlTuiError
    ├── output/
    │   ├── mod.rs             Output
    │   ├── error.rs           TowlOutputError
    │   ├── formatter/
    │   │   ├── mod.rs         FormatterImpl
    │   │   ├── error.rs       FormatterError
    │   │   └── formatters/
    │   │       ├── mod.rs     Formatter dispatch
    │   │       ├── csv.rs
    │   │       ├── json.rs
    │   │       ├── markdown.rs
    │   │       ├── table.rs
    │   │       └── toml.rs
    │   └── writer/
    │       ├── mod.rs         WriterImpl
    │       ├── error.rs       WriterError
    │       └── writers/
    │           ├── file.rs    FileWriter
    │           └── stdout.rs  StdoutWriter
    └── error/
        └── mod.rs             TowlError

tests/
├── integration/               Integration tests
├── property/                  Property-based tests
└── fixtures/                  Test fixtures

Dependencies

Security

towl applies defence-in-depth across configuration, scanning, and output.

Path Traversal Protection

All user-supplied paths are checked for .. components before use:

• Config paths -- towl init --path rejects traversal attempts
• Scan paths -- towl scan <path> validates before walking
• Output paths -- -o <path> is validated and symlinks are resolved

The check uses contains_path_traversal() which inspects each path component for ...

Symlink Resolution

Output file paths are resolved via std::fs::canonicalize() before writing. This prevents symlink-based escape from the intended output directory.

Resource Limits

Hard limits prevent denial-of-service via large repositories or malicious inputs:

Secret Handling

The GitHub token (TOWL_GITHUB_TOKEN) is:

• Never stored in config files -- Only accepted via environment variable
• Stored as SecretString -- Uses the secrecy crate
• Masked in debug output -- Debug and Display show [REDACTED]
• Zeroed on drop -- Memory is cleared when the config is dropped

Environment Variable Restriction

Eight environment variables are read:

Secrets (TOWL_GITHUB_TOKEN, TOWL_LLM_API_KEY) are stored as SecretString and never written to config files or logs. No other environment variables influence behaviour.

Config File Safety

• --force required for overwriting existing config files
• Pattern array limits -- Each pattern field is capped at 100 entries
• Pattern length limits -- Individual regex patterns capped at 256 characters
• TOML parsing -- Uses config crate with serde for type-safe deserialization

Git Integration

• Git operations use tokio::process::Command to run git as a subprocess
• Only read-only git commands are executed (git remote get-url origin)
• No git credentials are accessed or stored

LLM CLI Subprocess Safety

When using claude-code or codex providers, towl spawns CLI subprocesses:

• Command validation -- Relative paths containing .. or non-absolute / are rejected
• Timeout -- CLI processes are killed after 120 seconds
• Input via stdin -- Prompts are piped through stdin, not shell arguments (prevents injection)
• Stderr capture -- CLI error output is captured and included in error messages

.gitignore Respect

The ignore crate automatically respects .gitignore rules during directory walking, preventing scanning of files the user has excluded from version control.

Error Messages

Error messages include file paths and context for debugging but do not expose internal implementation details or sensitive data. The SecretString type ensures tokens cannot leak through error formatting.

Threat Model

CI/CD

towl uses GitHub Actions for continuous integration and documentation deployment.

Documentation Deployment

The docs.yml workflow builds and deploys the mdBook documentation to GitHub Pages.

Trigger: Pushes to main that modify files in docs/ or the workflow file itself. Can also be triggered manually via workflow_dispatch.

Pipeline:

1. Build -- Installs mdBook, runs mdbook build docs, uploads the docs/book/ directory as a Pages artifact
2. Deploy -- Deploys the artifact to GitHub Pages

Permissions required:

• contents: read -- Read repository files
• pages: write -- Deploy to GitHub Pages
• id-token: write -- Authenticate with Pages

The workflow uses concurrency control (group: pages, cancel-in-progress: true) to prevent overlapping deployments.

Running Locally

Build and preview the documentation locally:

# Install mdBook
cargo install mdbook

# Build docs
mdbook build docs

# Serve with live reload
mdbook serve docs

The built documentation is output to docs/book/.

Contributing

Getting Started

git clone https://github.com/glottologist/towl.git
cd towl
cargo build

Requirements

• Rust 1.75+ (see rust-toolchain.toml)
• git on PATH

Development Commands

# Build
cargo build

# Run all tests
cargo nextest run   # preferred
cargo test          # fallback

# Clippy (strict)
cargo clippy --all-targets --all-features

# Format
cargo fmt

# Run the binary
cargo run -- scan
cargo run -- scan -f json -o todos.json
cargo run -- config
cargo run -- init

Project Structure

See Architecture for a full layout. Key entry points:

• src/bin/towl.rs -- CLI binary
• src/lib/mod.rs -- Library root
• tests/ -- Integration and property-based tests

Testing

Test Hierarchy

Tests follow a strict hierarchy:

1. proptest (property-based) -- First choice for pure functions, parsers, validators, serialisation roundtrips
2. rstest (parameterized) -- For specific known cases (< 10 inputs with exact expected outputs)
3. Standalone -- Last resort, for complex integration scenarios

Running Tests

# All tests
cargo nextest run

# Specific module
cargo nextest run scanner

# Property-based tests only
cargo nextest run proptest

# Integration tests only
cargo nextest run --test '*'

Code Style

• Follow Rust naming conventions (snake_case for functions, CamelCase for types)
• All public items need doc comments (///)
• No #[allow(...)] attributes -- fix the underlying issue
• No .unwrap() / .expect() in production code -- use ? with typed errors
• No as numeric casts -- use try_from / into / From
• Minimise .clone() -- prefer borrowing, see Clone Reduction Policy

Error Handling

• Use thiserror for error type derivation
• Each module defines its own error enum
• Errors propagate upward via ? and #[from]
• Never silently discard Result values

Adding a New Output Format

1. Create src/lib/output/formatter/formatters/yourformat.rs
2. Implement the Formatter trait
3. Add a variant to FormatterImpl in formatters/mod.rs
4. Add dispatch in FormatterImpl::format()
5. Add a variant to OutputFormat in src/lib/cli/mod.rs
6. Update the format-to-writer mapping in Output::new()
7. Add tests (proptest for roundtrips, rstest for edge cases)

Adding a New TODO Type

1. Add a variant to TodoType in src/lib/comment/todo.rs
2. Update Display, TryFrom<&str>, as_filter_str()
3. Add a default pattern to default_todo_patterns() in src/lib/config/types.rs
4. Add a pattern mapping in the parser
5. Update tests

Pull Requests

• Keep PRs focused on a single change
• Include tests for new functionality
• Ensure cargo clippy passes with zero warnings
• Ensure cargo fmt produces no changes
• Ensure all tests pass