# qqrl — Quick Query for RustLedger

Rewrite of [ledger2bql](https://github.com/alensiljak/ledger2bql) as a fast, single-binary Rust CLI that delegates all ledger parsing and BQL execution to an externally installed `rledger` binary.

---

## Goals

- Translate Ledger CLI query syntax into BQL, execute via `rledger query`, format output
- No Python runtime required
- Fast startup (~15ms vs ~1-2s for current Python stack)
- Single static binary, cross-compiles to Android ARM64 for Termux
- Depends on `rledger` being available on `$PATH` — the user manages the rledger version independently
- License: AGPL-3.0

---

## Architecture

```txt
User input (CLI args)
        │
        ▼
qqrl (Rust CLI — clap)
  ├─ parse CLI args
  ├─ build BQL query string  (same logic as current Python)
  ├─ spawn: rledger query -f json $LEDGER_FILE "QUERY"
  ├─ parse JSON response
  ├─ format_output()  (running totals, hierarchy, lot arithmetic, etc.)
  └─ tabulate → stdout  (pager skipped)
```

`rledger` handles: ledger file parsing, BQL execution, all financial arithmetic.
`qqrl` handles: CLI UX, BQL query construction, output formatting/tabulation.

---

## Design Decisions

### External binary, not linked library

- User upgrades `rledger` independently via their package manager (brew, scoop, cargo, pkg)
- qqrl has no compile-time dependency on rledger internals — no Cargo.lock coupling
- Maintenance burden stays low after initial rewrite; improvements in speed/correctness come from rledger  
- Acceptable tradeoff: ~10-30ms subprocess overhead on Linux/Termux

### JSON output format

- `rledger query -f json FILE "QUERY"` returns structured data
- Avoids brittle text parsing; JSON schema is more stable than text table layout
- Must validate exact JSON schema in Phase 0 (field names, how Amount/Position are serialized)

### BQL query strings unchanged

- All `SELECT ... WHERE ... GROUP BY ... ORDER BY ... LIMIT` strings generated by the current Python code are valid BQL and run unchanged on rledger
- This is the key insight: only the I/O layer changes, not the query logic

---

## Commands

All 6 commands from ledger2bql are carried over:

| Command  | Alias | Description                           |
|----------|-------|---------------------------------------|
| `bal`    | `b`   | Account balances                      |
| `reg`    | `r`   | Transaction register                  |
| `query`  | `q`   | Execute named queries from .bean file |
| `lots`   | `l`   | Investment lots / cost basis          |
| `assert` | `a`   | Balance assertions                    |
| `price`  | `p`   | Price history                         |

Common options (all commands): `--begin`, `--end`, `--date-range`, `--sort`, `--limit`,
`--amount`, `--currency`, `--exchange`, `--total`

### Configuration

- `LEDGER_FILE` env var (or `$BEANCOUNT_FILE` for backward compat) — path to .bean file
- `.env` file support via `dotenvy` crate
- `RLEDGER_BIN` env var to override the rledger binary path (default: `rledger`)

---

## Phased Plan

### Phase 0: Compatibility Audit — *3-5 days, no code written*

Validate rledger behavior before committing to any implementation.

1. Install rledger, run `rledger query -f json tests/sample-ledger.bean "..."` for each query pattern used in ledger2bql (all 6 commands)
2. Document exact JSON schema — field names, how `Amount`, `Position`, `Inventory` are serialized
3. Test these specifically (high risk — not in rledger BQL docs):
   - `convert(position, 'EUR')` — used by `--exchange/-X`
   - `value(position)` — used by `lots`
   - `cost_number` column — used by `lots`
   - `#balances` system table — used by `assert`
   - `#prices` system table — used by `price`
4. Check Android ARM64 release binary on GitHub Releases, or test `cargo install rustledger` in Termux
5. For any missing feature: either find the rledger equivalent syntax, or file an issue upstream
6. **Gate**: do not start Phase 1 until all 6 commands have a confirmed working query pattern

#### Phase 0 results

The queries from `/tests/phase0-test.cmd` have been executed with the following results:

```sh
> rledger query -f json tests/sample-ledger.bean "SELECT date, currency, amount FROM #prices"
> rledger query -f json tests/sample-ledger.bean "SELECT date, account, amount FROM #balances"

error: failed to parse query: parse error at position 35: found '.' expected identifie', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-', '"', ''', '*', ' ', ' ', '
', ';', or end of input

> rledger query -f json tests/sample-ledger.bean "SELECT account, sum(convert(position, 'EUR')) GROUP BY account"

{
  "columns": [
    "account",
    "sum"
  ],
  "row_count": 16,
  "rows": [
    {
      "account": "Assets:Bank:Bank03581",
      "sum": {
        "positions": [
          {
            "currency": "EUR",
            "number": "3194.1000"
          }
        ]
      }
    },
...

> rledger query -f json tests/sample-ledger.bean "SELECT account, cost_number WHERE cost_number IS NOT NULL LIMIT 5"

{
  "columns": [
    "account",
    "cost_number"
  ],
  "row_count": 4,
  "rows": [
    {
      "account": "Equity:Stocks",
      "cost_number": "1.25"
    },
...

> rledger query -f json tests/sample-ledger.bean "SELECT account, value(SUM(position)) GROUP BY account"

{
  "columns": [
    "account",
    "value"
  ],
  "row_count": 16,
  "rows": [
    {
      "account": "Assets:Bank:Bank03581",
      "value": {
        "positions": [
          {
            "currency": "CHF",
            "number": "3000"
          }
        ]
      }
    },
...
```

#### Revised Plan Based on Phase 0 Results

**Summary**: 4 of 6 commands are unblocked for Phase 1 work. The `lots` command is now implemented in qqrl for the supported `rledger` query shapes. The `assert` and `price` commands still depend on rledger's `#balances` and `#prices` system tables, which are pending feature requests.

**Unblocked commands** (implement in Phase 1):

- `bal` (balance) — uses standard BQL, `convert(position, 'EUR')` works ✅
- `reg` (register) — uses standard BQL ✅
- `query` (named queries) — regex scan of .bean file ✅
- `lots` (investment lots) — core command implemented; active lots, `--all`, `--average`, filters, sorting, and limit work ✅

**Blocked commands** (Phase 1.5 or Phase 2, pending rledger):

- `assert` (balance assertions) — blocked on `#balances` system table ⏳
- `price` (price history) — blocked on `#prices` system table ⏳

**Partially blocked behavior inside implemented commands**:

- `lots` market value column — still blocked by current `rledger` `value(...)` behavior ⏳

**Phase 1 Strategy**:

1. Implement full CLI skeleton with all 6 commands defined
2. Implement the 4 unblocked commands (bal, reg, query, lots)
3. Add placeholder error messages for `assert` and `price` that point to the rledger feature request
4. Ship Phase 1 with 4/6 commands working, noting the reduced `lots` output until `rledger` value support is usable

When rledger releases `#balances` and `#prices` support, implement Phase 1.5 to complete the remaining 2 commands (no architectural changes needed).

When `rledger` exposes a stable `value(...)` query shape for lots, extend `qqrl lots` to restore the market-value column without changing the CLI surface.

**Revised timeline**: ~3–4 weeks for Phase 1 (instead of 4–6 weeks)

---

### Phase 1: Rust CLI — *~3-4 weeks*

Full rewrite. No intermediate Python step — go straight to Rust.

#### Project setup

- `cargo new qqrl`
- Dependencies: `clap` (CLI), `serde_json` (JSON parsing), `dotenvy` (.env), `comfy-table` (tabulation)
- `AGPL-3.0` license

#### Core infrastructure

(start here, everything else depends on it)

1. `src/config.rs` — resolve `LEDGER_FILE`, `RLEDGER_BIN`, `.env` loading
2. `src/runner.rs` — `run_bql_query(query: &str) -> Result<Vec<serde_json::Value>>` — spawns `rledger query -f json`, captures stdout, parses JSON, maps errors
3. `src/date_parser.rs` — port of `date_parser.py`: `parse_date()`, `parse_date_range()`
4. `src/utils.rs` — port of common utils: `parse_account_pattern()`, `parse_amount_filter()`, `parse_account_params()`
5. `src/cli.rs` — clap command definitions, shared options struct

#### Migrate Commands

(can be worked in parallel after infrastructure is done)

6. `src/commands/balance.rs` — `parse_query()` + `format_output()` including `--hierarchy` mode
7. `src/commands/register.rs` — `parse_query()` + `format_output()` with running totals
8. `src/commands/query.rs` — regex scan of .bean file for `query "name" "..."` directives (replaces `beancount.loader.load_file()`)
9. `src/commands/lots.rs` — `parse_query()` + `format_output()` — most complex; decimal arithmetic, depends on Phase 0 findings
10. `src/commands/assert.rs` — `parse_query()` + `format_output()`
11. `src/commands/price.rs` — `parse_query()` + `format_output()`

### Phase 1 status snapshot

- `balance` implemented and covered by integration tests
- `register` implemented and covered by integration tests
- `query` implemented
- `lots` implemented with a reduced output shape that omits market value for now
- `assert` placeholder implemented
- `price` placeholder implemented

### Lots status

Implemented in qqrl:

- dedicated `lots` CLI options: `--all`, `--average`, `--sort-by`
- default active-lot view using grouped positions with positive remaining quantity
- detailed lot view with `--all`
- average-cost view with `--average`
- account, date, currency, and amount filters
- sort and limit support
- integration coverage in `tests/lots_tests.rs`
- market value / `Value` column in output
- any exchange-driven valuation for lots

## Tests

- Port the existing pytest suite to Rust integration tests
- Each test: build expected BQL query → run against `sample-ledger.bean` via rledger → assert formatted output
- Use the same `tests/sample-ledger.bean` data file

## Release

- `cargo build --release`
- Cross-compile for Termux: `CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=... cargo build --release --target aarch64-unknown-linux-gnu`
- GitHub Actions CI for Linux x86_64, macOS, Windows, Linux aarch64

---

## Complexity by Command

| Command  | Query Building          | Output/JSON Parsing                                                   | Risk     |
|----------|-------------------------|-----------------------------------------------------------------------|----------|
| `bal`    | Easy                    | Medium — hierarchy mode needs parent-account aggregation              | Low      |
| `reg`    | Easy                    | Medium — running totals, multi-currency                               | Low      |
| `query`  | Easy — .bean regex scan | Easy                                                                  | Low      |
| `lots`   | Medium                  | High — mixed JSON shapes, active lot aggregation, market value output | **High** |
| `assert` | Easy                    | Easy                                                                  | Low      |
| `price`  | Easy                    | Easy                                                                  | Low      |

---

## Known Risks

| Risk                                          | Impact                             | Mitigation                                                                                      |
|-----------------------------------------------|------------------------------------|-------------------------------------------------------------------------------------------------|
| `convert(position, 'EUR')` not in rledger     | `--exchange/-X` broken             | Phase 0 test; workaround with two queries + post-processing, or file rledger issue              |
| `value(position)` not in usable rledger shape | `lots` market value column broken  | Current implementation omits the value column until upstream behavior is stable                 |
| `cost_number` column not in rledger           | `lots` cost per unit broken        | Phase 0 test; find rledger equivalent                                                           |
| `#balances` / `#prices` tables not supported  | `assert` / `price` commands broken | Phase 0 test; rledger uses `BALANCES` shorthand — find exact equivalent                         |
| rledger JSON schema changes                   | Output parser breaks silently      | Pin rledger minimum version in docs; test on CI with rledger installed                          |
| rledger not on `$PATH`                        | All commands fail                  | Clear error message: "rledger not found. Install from https://github.com/rustledger/rustledger" |

---

## Rust Crate Reference

| Crate                        | Purpose                                             |
|------------------------------|-----------------------------------------------------|
| `clap`                       | CLI argument parsing                                |
| `serde_json`                 | JSON parsing from rledger output                    |
| `dotenvy`                    | .env file loading                                   |
| `comfy-table`                | Output tabulation                                   |
| `regex`                      | Account pattern matching, .bean file query scanning |
| `rust_decimal`               | Decimal arithmetic for running totals, lots         |
| `chrono`                     | Date parsing and arithmetic                         |
| `thiserror`                  | Error types                                         |

---

## What Carries Over Unchanged

From ledger2bql, these things translate directly:

- All BQL query strings (identical SQL dialect)
- Date parsing logic (`date_parser.py` → `date_parser.rs`, pure logic)
- Account pattern handling (`^pattern`, `pattern$`, raw regex)
- Amount filter parsing (`>100EUR`, `=50USD`, etc.)
- `--hierarchy` aggregation algorithm
- Running total tracking per currency in `reg`
- Named query regex pattern for `.bean` files

---

## What Changes

- `beanquery.connect().execute()` → `std::process::Command` spawning `rledger`
- Python beancount object access (`.units.number`, `.currency`) → `serde_json::Value` field access
- `beancount.loader.load_file()` → regex scan of `.bean` file text
- `tabulate()` → `comfy-table` or equivalent Rust crate
- `click` → `clap`
- `loguru` → `tracing` (optional, for `--verbose`)
- `dotenv` → `dotenvy`

---

## rledger Resources

- RustLedger [repo](https://github.com/rustledger/rustledger)
- BQL Reference, [docs](https://rustledger.github.io/reference/bql.html)
- Query command, [doc](https://rustledger.github.io/commands/query.html)
- Compatibility [report](https://rustledger.github.io/reference/compatibility.html)
- RustLedger [releases](https://github.com/rustledger/rustledger/releases/latest) (binaries)
