# lispexp reads UTF-8 only; non-UTF-8 decoding is a consumer/boundary concern

lispexp accepts `&str` and therefore UTF-8 only; reading legacy non-UTF-8 files (e.g. latin-1 / euc-jp / shift_jis Emacs Lisp) is an explicit non-goal. This falls directly out of the zero-copy borrowed tree (ADR-0008): `Datum`/`Token` borrow `&str` slices, and `&str` is UTF-8 by definition — making the core operate on `&[u8]` instead would touch every type, hand consumers bytes instead of `str`, and still not answer "which encoding are these bytes," so it is rejected. Instead, transcoding is done at the boundary by the consumer/application before handing text to lispexp: detect the encoding (e.g. Emacs's `-*- coding: … -*-` cookie) and decode to UTF-8 (e.g. via `encoding_rs`, which is pure Rust and cross-compiles cleanly), or use `String::from_utf8_lossy` when only structure matters. Two facts make this cheap in practice: pure-ASCII legacy files are already valid UTF-8 and parse today (all 610 chibi-scheme corpus files did), so only files with real high-bit content in a non-UTF-8 encoding are affected — a low-frequency case — and after boundary transcoding the entire existing reader works unchanged, at the cost that spans then index the decoded text rather than the original bytes. If demand appears, an optional `encoding` cargo feature could ship a `decode_to_utf8(bytes) -> Cow<str>` helper, keeping the core dependency-free.
