# ecma-lex-cat

ECMAScript lexer producing `Spanned<Token>` values that downstream crates ([ecma-parse-cat](https://crates.io/crates/ecma-parse-cat), [boa-cat](https://crates.io/crates/boa-cat)) feed into [ecma-syntax-cat](https://crates.io/crates/ecma-syntax-cat).

The second layer of a multi-crate comp-cat-rs reformulation of a JavaScript engine targeting Tauri integration.

## Coverage

- All ES2024 keywords (37 reserved + strict-reserved) and contextual identifiers.
- All punctuators and operators (~50 distinct tokens).
- All numeric literal forms: decimal, hex (`0x`), octal (`0o`), binary (`0b`), exponential, BigInt suffix (`123n`).
- String literals with full escape handling: `\n \t \r \\ \' \" \0 \b \f \v \xNN \uNNNN \u{...}`.
- Template literals with `${}` interpolation; emitted as a sequence of `TemplateHead`, expression tokens, `TemplateMiddle`, `TemplateTail` (or `TemplateNoSubstitution` for the no-interpolation case).
- Regex literals with last-token disambiguation (`/` is division after expressions; regex after operators).
- Line and block comments (skipped, not tokenised).

Deferred to v0.2:

- Non-ASCII Unicode identifier characters.
- Numeric separators (`1_000_000`).
- Hashbang `#!` lines.
- Legacy octal `0n...` literals (rejected as `InvalidNumber`).

## Usage

```rust
# fn main() -> Result<(), ecma_lex_cat::error::Error> {
use ecma_lex_cat::lex;
use ecma_lex_cat::token::TokenKind;

let tokens = lex("let x = 1 + 2;")?;
// tokens[0].value() == &TokenKind::KwLet, etc.
# Ok(())
# }
```

## Building

```sh
cargo build
cargo test
RUSTFLAGS="-D warnings" cargo clippy --all-targets
```

## License

Licensed under either of [Apache License, Version 2.0](LICENSE-APACHE) or [MIT license](LICENSE-MIT) at your option.
