# TP-Lib: Train Positioning Library

[![CI](https://github.com/matdata-eu/tp-lib/actions/workflows/ci.yml/badge.svg)](https://github.com/matdata-eu/tp-lib/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/matdata-eu/tp-lib/branch/main/graph/badge.svg)](https://codecov.io/gh/matdata-eu/tp-lib)
[![crates.io](https://img.shields.io/crates/v/tp-lib-core.svg)](https://crates.io/crates/tp-lib-core)
[![PyPI](https://img.shields.io/pypi/v/tp-lib.svg)](https://pypi.org/project/tp-lib/)
[![NuGet](https://img.shields.io/nuget/v/TpLib.svg)](https://www.nuget.org/packages/TpLib/)
[![Documentation](https://img.shields.io/badge/docs-github.io-blue)](https://matdata-eu.github.io/tp-lib/)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)

**Status**: Under construction

Train positioning library excels in post-processing the GNSS positions of your measurement train to achieve an unambiguous network location. This library is your map matching assistant specifically for railway.

## Features

- 🚄 **High Performance**: R-tree spatial indexing for O(log n) nearest-track search
- 📍 **Accurate Projection**: Haversine distance and geodesic calculations with geo-rs
- 🛤️ **Train Path Calculation**: Probabilistic path calculation through rail networks using topology
- 🗺️ **Interactive Path Review**: Browser-based map webapp to visually review and edit calculated paths before projection
- 🌍 **CRS Aware**: Explicit coordinate reference system handling (EPSG codes)
- ⏰ **Timezone Support**: RFC3339 timestamps with explicit timezone offsets; timezone-less ISO 8601 datetimes assumed UTC
- 📊 **Multiple Formats**: CSV and GeoJSON input/output
- 🧪 **Well Tested**: 460 comprehensive tests (all passing) - unit, integration, contract, CLI, and doctests
- ⚡ **Production Ready**: Full CLI interface with validation and error handling

## Train Path Calculation

The library supports continuous train path calculation using network topology (netrelations) 
to determine the most probable route a train took through the railway network.

### How It Works

1. **Candidate Selection**: Find candidate track segments within cutoff distance of each GNSS position
2. **Emission Probability**: Calculate per-position probability using exponential decay of distance and heading alignment
3. **Viterbi Decoding**: Decode the globally optimal netelement sequence using a log-space Viterbi algorithm (HMM map matching, Newson & Krumm 2009). Transition probabilities are derived from shortest-path vs. great-circle distance through the topology graph. Bridge netelements are inserted automatically for path continuity.

### CLI Commands

```bash
# Default: Calculate path AND project coordinates onto it
tp-cli --gnss positions.csv --network network.geojson --output result.csv

# Calculate path only (export path segments without projection)
tp-cli calculate-path --gnss positions.csv --network network.geojson --output path.csv

# Simple projection (legacy mode, no path calculation)
tp-cli simple-projection --gnss positions.csv --network network.geojson --output projected.csv

# Use pre-calculated path for projection
tp-cli --gnss positions.csv --network network.geojson --train-path path.csv --output result.csv

# Review calculated path in browser before projection (integrated mode)
tp-cli --gnss positions.csv --network network.geojson --output result.csv --review

# Launch standalone webapp to review/edit a path file
tp-cli webapp --network network.geojson --train-path path.csv --output reviewed_path.csv
```

### Debug Output

Pass `--debug` to write intermediate HMM calculation results as GeoJSON files to a `debug/` subdirectory next to the output file.  
See **[DEBUG.md](DEBUG.md)** for a full description of the debug output files, their properties, and a typical debugging workflow.

### Algorithm Parameters

| Parameter | Default | Description |
|-----------|---------|-------------|
| `--distance-scale` | 10.0 | Distance decay scale (meters) |
| `--heading-scale` | 2.0 | Heading decay scale (degrees) |
| `--cutoff-distance` | 500.0 | Max distance for candidate selection (meters) |
| `--heading-cutoff` | 10.0 | Max heading difference to accept (degrees) |
| `--probability-threshold` | 0.02 | Minimum emission probability for candidate inclusion |
| `--max-candidates` | 3 | Max candidates per GNSS position |
| `--resampling-distance` | None | Optional GNSS resampling distance (meters) |

### Performance

- **Benchmark**: 1000 positions × 50 netelements in ~900μs (target was <10s)
- **Scalability**: Sub-linear scaling with R-tree spatial indexing
- **Memory**: Handles 10,000+ positions efficiently

## Train Path Review Webapp

The library ships a companion browser-based map webapp (`tp-webapp`) that lets you visually inspect and edit a calculated train path before using it for projection. No npm, no Node.js, no frontend build step required — the web assets are embedded in the binary at compile time using `rust-embed`.

### Modes

| Mode | Command | Use case |
|------|---------|----------|
| **Standalone** | `tp-cli webapp` | Review/edit an existing path file; save result to disk; server stays alive |
| **Integrated** | `tp-cli … --review` | Pause the projection pipeline after path calculation; confirm or abort in the browser |

### What You Can Do in the Browser

- View all network segments on a Leaflet map (OpenStreetMap basemap, toggleable)
- Path segments are highlighted with a colour-coded confidence scale
- **Add** a netelement to the path by clicking it on the map — snapped to the correct topological position
- **Remove** a path segment by clicking it — segment reverts to the default style
- Sidebar shows an ordered list of segments with probability scores
- **Dark mode** toggle (auto-activates from OS `prefers-color-scheme`)
- **Close Tab** button for clean exit after the server shuts down

### Standalone Usage

```sh
tp-cli webapp \
  --network network.geojson \
  --train-path path.csv \
  --output reviewed_path.csv
```

A browser opens at `http://127.0.0.1:8765`. Click **Save** to write the reviewed path; the server stays alive for further edits. Press Ctrl+C when done.

The saved file is accepted directly by `--train-path`:

```sh
tp-cli --gnss positions.csv --network network.geojson \
  --train-path reviewed_path.csv --output result.csv
```

### Integrated Usage

```sh
tp-cli --gnss positions.csv --network network.geojson \
  --output result.csv --review
```

1. CLI calculates the train path
2. Browser opens automatically — review and edit the path
3. Click **Confirm** → path artifact saved as `result-path.csv`; projection continues
4. Click **Abort** → CLI exits with non-zero code

### Build Without Web Server

```sh
cargo build --package tp-cli --no-default-features
```

## Project Structure

```
tp-lib/                    # Rust workspace root
├── tp-core/               # Core Rust library
│   ├── src/
│   │   ├── models/        # Data models (GnssPosition, Netelement, ProjectedPosition, PathOrigin)
│   │   ├── projection/    # Projection algorithms (geom, spatial indexing)
│   │   ├── path/          # Train path calculation (candidate, probability, graph, viterbi)
│   │   ├── io/            # Input/output (CSV, GeoJSON, Arrow)
│   │   ├── crs/           # Coordinate reference system transformations
│   │   ├── temporal/      # Timezone handling utilities
│   │   └── errors.rs      # Error types
│   ├── tests/
│   │   ├── unit/          # Unit tests
│   │   └── integration/   # Integration tests
│   └── benches/           # Performance benchmarks
├── tp-cli/                # Command-line interface
├── tp-webapp/             # Interactive path review web server (axum + Leaflet.js)
├── tp-py/                 # Python bindings (PyO3)
└── tp-net/                # .NET bindings (csbindgen + System.Text.Json)
```

## Quick Start

### Prerequisites

- Rust 1.91.1+ (install from [rustup.rs](https://rustup.rs/))
- Python 3.12+ (for Python bindings)

### Build from Source

```bash
# Clone repository
git clone https://github.com/matdata-eu/tp-lib
cd tp-lib

# Build all crates
cargo build --workspace

# Run tests
cargo test --workspace

# Run benchmarks
cargo bench --workspace
```

### Docker Usage

#### Production Deployment

Use Docker to run the CLI without installing Rust:

```bash
# Build production image
docker build -t tp-lib:latest .

# Run with mounted data directory
docker run --rm -v $(pwd)/data:/data tp-lib:latest \
  --gnss-file /data/gnss.csv \
  --crs EPSG:4326 \
  --network-file /data/network.geojson \
  --output-format csv > output.csv

# Or use docker-compose
docker-compose up tp-cli
```

#### Running Tests in Docker

Run the complete test suite including CRS transformation tests:

```bash
# Using docker-compose (recommended)
docker-compose run --rm test

# Or build and run test image directly
docker build -f Dockerfile.test -t tp-lib-test .
docker run --rm tp-lib-test

# Run specific tests
docker-compose run --rm test cargo test test_identity_transform

# Run only CRS transformation tests
docker-compose run --rm test cargo test crs_transform

# Interactive shell for debugging
docker run --rm -it tp-lib-test bash
```

**Why Docker for tests?**

- **Complete test coverage**: Runs all tests including CRS transformation tests
- **Consistent environment**: Same Rust version across all machines
- **No local setup needed**: No need to install Rust toolchain locally
- **CI/CD ready**: Use `Dockerfile.test` in GitHub Actions or other CI systems

### Usage Examples

```bash
# CLI usage - CSV input/output
tp-cli --gnss-file train_positions.csv \
       --crs EPSG:4326 \
       --network-file railway_network.geojson \
       --output-format csv > projected.csv

# GeoJSON output with custom warning threshold
tp-cli --gnss-file positions.csv \
       --crs EPSG:4326 \
       --network-file network.geojson \
       --output-format json \
       --warning-threshold 100.0 > projected.geojson

# Custom CSV column names
tp-cli --gnss-file data.csv \
       --crs EPSG:4326 \
       --network-file network.geojson \
       --lat-col lat --lon-col lon --time-col timestamp
```

### Library Usage

```rust
use tp_lib_core::{parse_gnss_csv, parse_network_geojson, RailwayNetwork};
use tp_lib_core::{project_gnss, ProjectionConfig};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Load railway network from GeoJSON
    let netelements = parse_network_geojson("network.geojson")?;
    let network = RailwayNetwork::new(netelements)?;

    // Load GNSS positions from CSV
    let positions = parse_gnss_csv(
        "gnss.csv",
        "EPSG:4326",
        "latitude",
        "longitude",
        "timestamp"
    )?;

    // Project onto network with default config (50m warning threshold)
    let config = ProjectionConfig::default();
    let projected = project_gnss(&positions, &network, &config)?;

    // Use results
    for pos in projected {
        println!(
            "Position at {}m on netelement {} (accuracy: {:.2}m)",
            pos.measure_meters,
            pos.netelement_id,
            pos.projection_distance_meters
        );
    }

    Ok(())
}
```

## Implementation Notes

### Performance

- **Target**: < 10 seconds for 1000 GNSS positions × 50 netelements (SC-001)
- **Memory**: Handles 10,000+ positions without exhaustion (SC-006)
- **Accuracy**: 95% of positions within 2m projection distance (GPS quality dependent)
- **R-tree Complexity**: O(log n) nearest-neighbor search

### Input Data Requirements

**GNSS CSV:**

```csv
latitude,longitude,timestamp,altitude,hdop
50.8503,4.3517,2025-12-09T14:30:00+01:00,100.0,2.0
```

- RFC3339 timestamps with timezone (e.g. `2025-12-09T14:30:00+01:00` or `2025-12-09T14:30:00Z`); timezone-less ISO 8601 datetimes (e.g. `2025-12-09T14:30:00`) are accepted and assumed UTC
- CRS must be specified via `--crs` flag
- Column names configurable with `--lat-col`, `--lon-col`, `--time-col`

**Railway Network GeoJSON:**

```json
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": { "id": "NE001", "crs": "EPSG:4326" },
      "geometry": {
        "type": "LineString",
        "coordinates": [
          [4.35, 50.85],
          [4.36, 50.86]
        ]
      }
    }
  ]
}
```

- LineString geometries (track centerlines)
- Unique `id` property per netelement
- `crs` property with EPSG code

### Troubleshooting

**"Large projection distance" warnings:****

Indicates GNSS position is far from nearest track (> threshold). Possible causes:

- GPS inaccuracy or poor signal quality
- Train on parallel track not in network
- Missing netelement in railway network
- CRS mismatch between GNSS and network
- Track geometry outdated or incorrect

Adjust threshold with `--warning-threshold` flag or investigate data quality.

**"No Python 3.x interpreter found" build error:**

Building with default features requires Python for PyO3 bindings. Disable with:

```bash
cargo build --no-default-features
```

Or install Python 3.12+ and ensure it's in your PATH.

### Known Issues

1. **Windows Build Dependencies**: Requires MSVC toolchain or mingw-w64 for some native dependencies
2. **Python Bindings**: Requires Python 3.12+ installed (excluded from tp-py crate builds by default)

## CRS Transformations

TP-Lib uses **proj4rs**, a pure Rust implementation of PROJ.4, for coordinate reference system transformations. This eliminates system dependencies and enables cross-platform compatibility.

**Key Features:**

- **Pure Rust**: No external C libraries required (libproj, sqlite3, etc.)
- **Zero system dependencies**: Works on Windows, Linux, macOS without installation
- **EPSG support**: Uses `crs-definitions` crate for EPSG code lookup
- **WASM compatible**: Can be used in browser environments
- **Always enabled**: CRS transformations are available by default

**Supported Transformations:**

TP-Lib has been tested with Belgian railway coordinate systems:

- EPSG:4326 (WGS84) ↔ EPSG:31370 (Belgian Lambert 72)
- EPSG:4326 (WGS84) ↔ EPSG:3812 (Belgian Lambert 2008)
- Any EPSG codes supported by [crs-definitions](https://docs.rs/crs-definitions/)

**Usage:**

```rust
use tp_lib_core::crs::CrsTransformer;
use geo::Point;

// Create transformer (EPSG codes or PROJ strings)
let transformer = CrsTransformer::new(
    "EPSG:4326".to_string(),
    "EPSG:31370".to_string()
)?;

// Transform point (automatic degree/radian conversion)
let wgs84_point = Point::new(4.3517, 50.8503);
let lambert_point = transformer.transform(wgs84_point)?;
```

**Technical Details:**

- proj4rs automatically handles radian/degree conversions for geographic CRS
- EPSG codes are resolved to PROJ strings using the crs-definitions crate
- Custom PROJ strings can be used directly instead of EPSG codes
- Transformation accuracy matches PROJ for standard 2D transformations

**Limitations:**

- proj4rs implements PROJ.4 API (2D transformations only)
- No 3D/4D or orthometric transformations
- Grid shift support is experimental
- For complex geodetic requirements, consider using [PROJ](https://proj.org/) directly

## Documentation

### API Documentation

**Online:** https://matdata-eu.github.io/tp-lib/

The documentation is automatically built and deployed on every push to `main`. It includes:

- **tp-core**: Core library API with examples
- **tp-cli**: Command-line interface documentation
- **tp-py**: Python bindings API reference
- **tp-net**: .NET bindings API reference

**Build locally:**

```bash
# Generate documentation for all workspace crates
cargo doc --no-deps --workspace

# Open in browser (on Windows)
start target/doc/index.html

# Open in browser (on Linux/macOS)
open target/doc/index.html  # macOS
xdg-open target/doc/index.html  # Linux
```

### Specification Documents

**Feature 001 — GNSS Projection**
- [Feature Specification](specs/001-gnss-projection/spec.md)
- [Implementation Plan](specs/001-gnss-projection/plan.md)
- [Data Model](specs/001-gnss-projection/data-model.md)
- [CLI Contract](specs/001-gnss-projection/contracts/cli.md)
- [Tasks](specs/001-gnss-projection/tasks.md)

**Feature 002 — Train Path Calculation**
- [Feature Specification](specs/002-train-path-calculation/spec.md)
- [Implementation Plan](specs/002-train-path-calculation/plan.md)
- [Data Model](specs/002-train-path-calculation/data-model.md)
- [Tasks](specs/002-train-path-calculation/tasks.md)

**Feature 003 — Train Path Review Webapp**
- [Feature Specification](specs/003-path-review-webapp/spec.md)
- [Implementation Plan](specs/003-path-review-webapp/plan.md)
- [Data Model](specs/003-path-review-webapp/data-model.md)
- [REST API Contract](specs/003-path-review-webapp/contracts/api.md)
- [CLI Contract](specs/003-path-review-webapp/contracts/cli.md)
- [Quickstart](specs/003-path-review-webapp/quickstart.md)
- [Tasks](specs/003-path-review-webapp/tasks.md)

### CI/CD & Workflows

This project uses automated workflows for continuous integration and deployment:

- 🔄 **Continuous Integration**: Automated testing, linting, and security checks on every push
- 📦 **crates.io Publishing**: Automatic release to Rust package registry
- 🐍 **PyPI Publishing**: Automatic release to Python package index
- 📚 **Documentation Deployment**: Auto-deployed to GitHub Pages

See **[CI/CD Workflows Documentation](docs/WORKFLOWS.md)** for details on:
- Build and test automation
- Release process and version management
- Security and license validation
- Publishing to crates.io and PyPI
- Documentation deployment

### Constitution Compliance

This project follows the TP-Lib Constitution v1.1.0 principles:

- ✅ **I. Library-First**: Single unified library with quality external dependencies
- ✅ **II. CLI Mandatory**: Command-line interface for all functionality
- ✅ **III. High Performance**: Apache Arrow, R-tree spatial indexing
- ✅ **IV. TDD**: Test-driven development with FIRST TEST validation
- ✅ **V. Full Coverage**: Comprehensive test suite (unit, integration, property-based)
- ✅ **VI. Timezone Awareness**: DateTime<FixedOffset> for all timestamps
- ✅ **VII. CRS Explicit**: All coordinates include CRS specification
- ✅ **VIII. Error Handling**: Typed errors with thiserror, fail-fast validation
- ✅ **IX. Data Provenance**: Preserve original GNSS data, audit logging
- ✅ **X. Integration Flexibility**: Rust API + CLI + Python bindings + .NET bindings

## Contributing

This project follows strict TDD workflow:

1. Write test first (RED)
2. Implement minimum code to pass (GREEN)
3. Refactor while keeping tests green

See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

## License

Apache License 2.0 - See [LICENSE](LICENSE) for details

## Contact

TP-Lib Contributors - [GitHub Issues](https://github.com/Matdata-eu/tp-lib/issues)
