#+title: Developer workflow and release guidelines
#+options: toc:2 num:t

* Development setup

** Prerequisites

- Rust >= 1.88 (edition 2024)
- [[https://pixi.sh][pixi]] for environment management
- [[https://github.com/cocogitto/cocogitto][cocogitto]] (=cog=) for conventional commits and changelog

** Getting started

#+begin_src shell
git clone https://github.com/lode-org/readcon-core.git
cd readcon-core
pixi install

# Run tests
pixi r test

# Run with all features (needs capnproto)
pixi r test-all

# Build release
pixi r build
#+end_src

** Environment-specific workflows

#+begin_src shell
# Python bindings
pixi r -e python python-build
pixi r -e python python-test

# Julia bindings
pixi r -e julia julia-test

# Documentation
pixi r -e docs docbld
# Docs output in docs/build/
#+end_src

* Commit conventions

The project uses [[https://www.conventionalcommits.org/][conventional commits]] enforced by cocogitto.
The CI lint check rejects non-conforming commit messages.

Recognized commit types (from =cog.toml=):

| Type | Changelog section | Use for |
|-------+-------------------+----------------------------------------------|
| feat | Features | New user-facing functionality |
| enh | Enhancements | Improvements to existing features |
| fix | Bugfixes | Bug corrections |
| bug | Bugfixes | Alias for fix |
| doc | Documentation | Documentation-only changes |
| tst | Tests | Test additions or modifications |
| bld | Buildsystem | Build system, CI, dependency changes |
| maint | Maintenance | Refactoring, cleanup, internal changes |
| bench | Benchmarks | Benchmark additions or modifications |
| gen | Generated | Auto-generated file updates |
| dat | Data | Test data or resource changes |

Examples:

#+begin_src text
feat: add convel format support with optional velocity fields
enh: integrate fast-float2 for f64 parsing hot path
fix: handle empty velocity section in multi-frame files
doc: update tutorials with Julia convel example
bld: add CMakeLists.txt for cmake subproject use
tst: add roundtrip test for convel writer
#+end_src

* Branching and workflow

- =main= is the release branch. All PRs target =main=.
- Feature branches: =feat/<description>=
- Bugfix branches: =fix/<description>=
- Keep commits logical and atomic. Soft-reset and rebase as needed
  (never on =main=).
- The CI runs on every PR: tests, lint, coverage, and benchmark
  regression checks (via Criterion + critcmp + asv-perch).

* Testing

#+begin_src shell
# Core Rust tests
cargo test

# All features (parallel, rpc, python)
cargo test --all-features

# Meson build with valgrind leak checking
meson setup bbdir -Dwith_tests=True -Dwith_examples=True
meson test -C bbdir

# Benchmarks
cargo bench
# or: pixi r bench
#+end_src

Test data lives in =resources/test/=. Use the =test_case!= macro in
integration tests to locate test files.

* Continuous integration

** Workflows

| Workflow | File | Trigger | Purpose |
|---------------------------+----------------------------+--------------------+-----------------------------------------------|
| Python wheels | =python_wheels.yml= | =v*= tag, PR | Build wheels for 5 platforms, publish to PyPI |
| Benchmark PR | =ci_benchmark.yml= | PR to main | Run Criterion benchmarks on base and PR |
| Comment benchmark results | =ci_bench_commenter.yml= | benchmark complete | Post comparison table as PR comment |
| Coverage | =coverage.yml= | push, PR | Code coverage via tarpaulin |
| Lint | =lint.yml= | push, PR | Conventional commit check + large file audit |

** Benchmark regression detection

The project uses a two-workflow pattern for safe benchmark PR comments:

1. =ci_benchmark.yml= runs =cargo bench= on both the base commit and the
   PR head in parallel (matrix strategy). It saves Criterion baselines as
   artifacts.

2. =ci_bench_commenter.yml= triggers on =workflow_run= completion. It
   downloads the Criterion results, compares them with =critcmp=, and
   posts the comparison table as a PR comment using [[https://github.com/HaoZeke/asv-perch][asv-perch]].

The =workflow_run= split is required for fork PRs to have write access
for posting comments (GitHub security model).

Regressions above 10x trigger automatic conversion to draft PR.

* Release process

** Version files

Versions are tracked in six source files that must stay synchronized:

1. =Cargo.toml= (=version = "X.Y.Z"=)
2. =meson.build= (=version: 'X.Y.Z'=)
3. =pyproject.toml= (=version = "X.Y.Z"=)
4. =pixi.toml= (=version = "X.Y.Z"=)
5. =docs/source/conf.py= (=release = "X.Y.Z"=)
6. =src/lib.rs= (=assert_eq!(VERSION, "X.Y.Z")= in the version test)

** Complete release checklist

1. Ensure all tests pass and the branch is clean:

   #+begin_src shell
   cargo test
   cargo test --all-features
   pixi r -e python python-test
   #+end_src

2. Bump the version in all six files listed above.

3. Regenerate =Cargo.lock= after the version bump:

   #+begin_src shell
   cargo test
   #+end_src

4. Regenerate the README from =readme_src.org=:

   #+begin_src shell
   emacs --batch \
     --eval "(progn (require 'package) (package-initialize) (require 'ox-md) \
                     (find-file \"readme_src.org\") \
                     (org-export-to-file 'md \"README.md\"))"
   #+end_src

5. Regenerate =CHANGELOG.md= with =cog=.
   Do not hand-edit the generated release section.

   #+begin_src shell
   {
     sed -n '1,3p' CHANGELOG.md
     cog changelog
   } > /tmp/CHANGELOG.md
   mv /tmp/CHANGELOG.md CHANGELOG.md
   #+end_src

6. Commit the release preparation:

   #+begin_src shell
   git add Cargo.toml Cargo.lock meson.build pyproject.toml \
           pixi.toml docs/source/conf.py src/lib.rs \
           CHANGELOG.md README.md docs/orgmode/contributing.org
   git commit -m "maint: bump to vX.Y.Z"
   #+end_src

7. Tag the release with a signed tag:

   #+begin_src shell
   git tag -s vX.Y.Z -m "vX.Y.Z"
   #+end_src

8. Push to both remotes:

   #+begin_src shell
   git push origin main --tags
   git push upstream main --tags
   #+end_src

9. Publish to crates.io:

   #+begin_src shell
   cargo publish
   #+end_src

10. The =v*= tag push triggers the =python_wheels.yml= workflow which
   builds wheels for Linux (x86_64, aarch64), macOS (Intel, ARM), and
   Windows (x86_64), then publishes to PyPI via OIDC trusted publisher.

11. Create the GitHub release with artifacts:

    #+begin_src shell
    cargo build --release

    # Package shared/static library + headers for C/C++/Julia
    mkdir -p /tmp/pkg/{lib,include}
    cp target/release/libreadcon_core.so /tmp/pkg/lib/
    cp target/release/libreadcon_core.a /tmp/pkg/lib/
    cp include/readcon-core.h /tmp/pkg/include/
    cp include/readcon-core.hpp /tmp/pkg/include/
    tar czf readcon-core-vX.Y.Z-linux-x86_64.tar.gz -C /tmp pkg/

    # Package Julia bindings
    tar czf ReadCon.jl-vX.Y.Z.tar.gz -C julia ReadCon/

    # Create release (attach capnp schema too)
    gh release create vX.Y.Z \
      readcon-core-vX.Y.Z-linux-x86_64.tar.gz \
      ReadCon.jl-vX.Y.Z.tar.gz \
      schema/ReadCon.capnp \
      --repo lode-org/readcon-core \
      --title "vX.Y.Z" \
      --notes-file <(cog changelog --at vX.Y.Z)
    #+end_src

12. Verify all distribution channels:
    - [[https://crates.io/crates/readcon-core][crates.io]] shows the new version
    - [[https://pypi.org/project/readcon/][PyPI]] shows the new version (wait for CI)
    - [[https://github.com/lode-org/readcon-core/releases][GitHub Releases]] has the tarball, Julia package, and capnp schema

** Initial PyPI setup (first release only)

For the very first release, PyPI trusted publisher is not yet
configured. Publish manually:

#+begin_src shell
# Build wheels
maturin build --release --features python

# Upload (first time, creates the project on PyPI)
uvx twine upload target/wheels/*
#+end_src

Then configure trusted publisher on PyPI:

1. Go to https://pypi.org/manage/project/readcon/settings/publishing/
2. Add a new pending publisher:
   - Owner: =lode-org=
   - Repository: =readcon-core=
   - Workflow: =python_wheels.yml=
   - Environment: =pypi=

After this, all subsequent tagged releases auto-publish via OIDC.

* Adding new features

** Feature gates

Optional functionality is gated behind Cargo features:

| Feature | Dependencies | Purpose |
|----------+--------------------------------+--------------------------|
| parallel | rayon | Multi-frame parallel parse |
| rpc | capnp, capnp-rpc, tokio, etc. | Cap'n Proto RPC serving |
| python | pyo3 | Python bindings |

Add new optional features in =Cargo.toml= under =[features]=.

** Adding a new binding

1. Add the binding source under =src/= (e.g., =src/python.rs=) or as
   a separate package (e.g., =julia/ReadCon/=).
2. Gate behind a feature flag if it adds dependencies.
3. Add tests under =tests/= or the binding's own test directory.
4. Document in =docs/orgmode/bindings.org= and =docs/orgmode/tutorials.org=.
5. Add a pixi environment and tasks if applicable.

** Updating the C header

The C header =include/readcon-core.h= is generated by cbindgen.
After modifying =src/ffi.rs=:

#+begin_src shell
cbindgen --config cbindgen.toml --crate readcon --output include/readcon-core.h -- src/lib.rs
#+end_src

Or let the meson/cmake build regenerate it.

The C++ header =include/readcon-core.hpp= is hand-maintained and must
be updated manually to match any changes to =CAtom=, =CFrame=, or
the FFI function signatures.
