# cera-wasm

`wasm-bindgen` browser / Node bindings for the
[cera](https://github.com/hyeons-lab/cera) inference engine.

> Status: pre-1.0. Today's surface covers manifest parsing, model
> loading (CPU-only), engine metadata + capability probes, full
> tokenizer access (`encode` / `decode` / `applyChatTemplate` /
> `specialTokenId` / `isSpecialToken`), session lifecycle (`reset`
> / `cancel` / `clearCancel`), and sync streaming text generation
> via `Session.generate(opts, cb)`. `Session.appendAudio` is wired
> as a placeholder symbol — the cera-core method is still a
> scaffold (always errors); the wasm signature is locked in so JS
> code stabilizes against the same shape the JVM/Apple bindings
> expose.

## Install

```sh
npm install @hyeons-lab/cera-wasm  # not yet published
```

For now, download the artifact matching your consumer shape from
the latest [CI run](https://github.com/hyeons-lab/cera/actions/workflows/ci.yml)
on `main` — three are produced per build:

| Artifact | wasm-pack target | When to use |
|---|---|---|
| `cera-wasm-pkg-bundler` | `bundler` | webpack 5+, Vite, Rollup (with wasm plugin), Parcel — the typical app build |
| `cera-wasm-pkg-web` | `web` | `<script type="module">` direct in the browser, or any bundler-less ESM workflow |
| `cera-wasm-pkg-nodejs` | `nodejs` | `require('@hyeons-lab/cera-wasm')` from CommonJS Node, or older Node without ESM `import` |

```sh
npm install /path/to/downloaded/pkg-bundler  # or pkg-web / pkg-nodejs
```

> **One npm package, three target shapes:** all three artifacts
> ship `package.json.name` = `@hyeons-lab/cera-wasm` today. The
> publish workflow (deferred) will resolve the collision before
> shipping to the registry — likely by publishing each target as
> a separate scoped package (`@hyeons-lab/cera-wasm`,
> `@hyeons-lab/cera-wasm-web`, `@hyeons-lab/cera-wasm-nodejs`).

## Usage

The examples below assume the **`bundler`** target. The `web`
target also needs a one-time `await init()` call before the first
export — see the
[wasm-pack docs](https://rustwasm.github.io/docs/wasm-pack/tutorials/npm-browser-packages/getting-started.html)
for the init pattern. The `nodejs` target does **not** need an
explicit init: `require('@hyeons-lab/cera-wasm')` returns a ready
module (the entry self-loads the wasm via `fs.readFileSync`).

### Manifest parsing

```js
import { ceraVersion, Manifest } from '@hyeons-lab/cera-wasm';

console.log(ceraVersion());  // e.g. "0.1.0"

const res = await fetch('/path/to/manifest.json');
const bytes = new Uint8Array(await res.arrayBuffer());
const manifest = Manifest.parse(bytes);

console.log(manifest.inferenceType);   // "llama.cpp/text-to-text"
console.log(manifest.modelUrl);        // "https://.../model.gguf"
console.log(manifest.schemaVersion);   // "1.0.0"
```

### Loading a model + tokenizing

```js
import { CeraEngine } from '@hyeons-lab/cera-wasm';

// Fetch the GGUF (use the `modelUrl` from a parsed manifest, or a
// direct URL).
const res = await fetch('/path/to/model.gguf');
const bytes = new Uint8Array(await res.arrayBuffer());

// Construct the engine. Optional `contextSize` defaults to 4096.
// Backend is forced to CPU on wasm.
const engine = CeraEngine.fromGgufBytes(bytes, 2048);

console.log(engine.architecture);     // "lfm2"
console.log(engine.maxSeqLen);        // 2048 (clamped to min(contextSize, gguf max))
console.log(engine.contextSize);      // 2048 — what you passed (or 4096 if omitted)
console.log(engine.vocabSize);
console.log(engine.quantization);     // "Q4_0", "Q8_0", "BF16", etc.
console.log(engine.hasChatTemplate);  // true / false
console.log(engine.addBosToken);      // honor when hand-building token sequences

// Modality capability probe. Plain JS object — no .free() needed,
// destructurable. Today every model loaded via fromGgufBytes
// reports text-only because wasm uses cera's synthetic-text
// manifest path; a model-aware loader (planned) will surface real
// audioIn / imageIn flags.
const { textIn, audioIn, audioOut, imageIn } = engine.capabilities;

// Tokenize a string.
const tok = engine.tokenizer;
const ids = tok.encode('hello world');  // Uint32Array
console.log(ids);
console.log(tok.decode(ids));           // round-trips to "hello world"

// Look up a control token by literal vocab name (only tokens
// flagged with token_type 3 / 4 in GGUF metadata are reachable).
const imStart = tok.specialTokenId('<|im_start|>');  // number | undefined

// Filter control tokens from a streamed batch before rendering
// to UI — keeps `<|im_end|>` etc. out of the displayed text.
const visible = ids.filter(id => !tok.isSpecialToken(id));

// Render the GGUF-embedded Jinja chat template against a message
// list. `addGenerationPrompt` defaults to `true` for the
// "send-to-model-and-await-response" case.
if (engine.hasChatTemplate) {
    const prompt = tok.applyChatTemplate([
        { role: 'system', content: 'You are a helpful assistant.' },
        { role: 'user',   content: 'Hello!' },
    ]);
    console.log(prompt);
}

// Release the model bytes from wasm memory when done.
engine.free();
```

> **Memory note:** `CeraEngine` keeps the entire GGUF resident in
> wasm linear memory. Always `engine.free()` (or use the
> `[Symbol.dispose]()` pattern with `using` in TC39 explicit
> resource management) when you're done — otherwise the model
> stays alive until the page unloads.

### Inference (text)

```js
import { CeraEngine, SessionConfig, GenerateOpts } from '@hyeons-lab/cera-wasm';

const engine = CeraEngine.fromGgufBytes(gguf, 2048);
const tok = engine.tokenizer;
// `newSession` requires a SessionConfig. Pass `new SessionConfig()`
// for the cera defaults (random sampler seed, no n_keep pin, etc).
// See "Reproducibility (seeded sampler)" below for the knob list.
const session = engine.newSession(new SessionConfig());

// Seed the conversation. Use `session.appendText(prompt)` for the
// common case (tokenizer is invoked internally), or
// `session.appendTokens(ids)` when you need control over BOS/EOS.
session.appendText('Hello, what is the capital of France?');

// Configure decoding. All fields default to cera's native defaults
// (max 256 tokens, temperature 0.7, top-p 0.9, top-k 40, no stops,
// flush every 16 tokens or 50 ms).
const opts = new GenerateOpts();
opts.maxTokens = 64;
opts.temperature = 0.0;
// `tok.eosToken` is `number | undefined`. `new Uint32Array([undefined])`
// silently coerces to `0` — which would stop decoding the moment
// token 0 is produced. Always guard the lookup.
if (tok.eosToken != null) {
    opts.stopTokens = new Uint32Array([tok.eosToken]);
}

// Stream tokens as they decode. The callback fires per flush
// boundary (every `flushEveryTokens` decoded tokens, OR every
// `flushEveryMs` ms — whichever hits first) with just the *new*
// tokens, not the cumulative buffer.
let acc = [];
const summary = session.generate(opts, (newTokens) => {
    acc.push(...newTokens);
    process.stdout.write(tok.decode(newTokens));  // streaming text
});

console.log('\n---');
console.log('finish:', summary.finishReason);          // "Stop" | "MaxTokens" | ...
console.log('tokens:', summary.tokensGenerated);
console.log('decode ms:', summary.decodeMs);

session.free();
engine.free();
```

### Reproducibility (seeded sampler)

Pass a `SessionConfig` with a fixed `seed` to `newSession` so the
sampler RNG is deterministic across runs. Same seed + same prompt
+ same `GenerateOpts` → identical token sequence:

```js
import {
    CeraEngine,
    SessionConfig,
    GenerateOpts,
    TurboQuantConfig,
} from '@hyeons-lab/cera-wasm';

const cfg = new SessionConfig();
cfg.seed = 42n;        // BigInt — wasm-bindgen maps Rust u64 to JS BigInt

// You can also tune:
//   cfg.maxSeqLen = 1024;   // further lower the KV cap below the
//                           // engine's effective max
//                           // (= min(engine.contextSize, model.maxSeqLen)).
//                           // Setting a value above that effective max
//                           // does NOT raise it — re-construct the engine
//                           // with a larger contextSize for that.
//   cfg.nKeep = 16;         // pin the first 16 tokens across context shifts
//                           // (useful for keeping a system prompt resident)
//   cfg.ubatchSize = 256;   // smaller chunked-prefill batches give finer
//                           // session.cancel() checkpoints during long prompts

// Optional: turn on TurboQuant KV compression. Compresses keys to
// ~3 bits/elem and values to ~2 bits/elem (plus f16 norms per
// block). Pass an explicit seed so the per-layer Hadamard
// rotations are reproducible — paired with `cfg.seed` above this
// keeps the whole session bitwise-identical across runs.
//
// Caveats:
// - Only kicks in when the model's `head_dim` is a power of two.
//   cera logs a warning and falls back to f32 if not — no JS
//   error.
// - Don't combine with `cfg.nKeep > 0` (context-shift); cera
//   warns at session creation and ignores nKeep on overflow.
const tq = new TurboQuantConfig(1234n);  // ctor sets keys + values = true
// tq.keys = false;  // flip per-side toggles for debugging
cfg.kvCompression = tq;  // setter consumes `tq` — read back via getter to inspect

const session = engine.newSession(cfg);
session.appendText('once upon a time');
const opts = new GenerateOpts();
opts.maxTokens = 16;
opts.temperature = 1.0;  // non-greedy so the seed actually matters
const out = [];
session.generate(opts, (toks) => out.push(...toks));
// `out` is identical for any session built with the same seed + prompt + opts.
```

> **Worker note:** `Session.generate` is **synchronous** and blocks
> the thread it runs on for the full decode duration (potentially
> seconds). On the browser main thread that freezes the page —
> always call from a Web Worker:
>
> ```js
> // worker.js
> import { CeraEngine, SessionConfig, GenerateOpts } from '@hyeons-lab/cera-wasm';
> self.onmessage = async (ev) => {
>     const engine = CeraEngine.fromGgufBytes(ev.data.gguf);
>     const session = engine.newSession(new SessionConfig());
>     session.appendText(ev.data.prompt);
>     const opts = new GenerateOpts();
>     opts.maxTokens = 128;
>     session.generate(opts, (toks) => self.postMessage({ kind: 'tokens', toks }));
>     self.postMessage({ kind: 'done' });
> };
> ```
>
> On Node the sync call also blocks the JS event loop — libuv's
> background I/O thread pool keeps running, but JS callbacks (HTTP
> handlers, timers, etc.) queue up and don't fire until generate
> returns. For server processes that need to handle other requests
> during inference, run generate inside a `worker_threads` Worker.
> For one-off scripts the block is fine.

### Cancellation

`session.cancel()` flips an atomic that the decode loop checks at
every flush boundary; generation exits with `finishReason === "Cancelled"`
at the next checkpoint.

The catch: JS workers (web + `worker_threads`) are single-threaded.
While `generate()` is blocking, the worker's own message handlers
**cannot run** — a `postMessage({ kind: 'cancel' })` from the main
thread queues but doesn't dispatch until `generate` returns. So a
plain JS flag set by an `onmessage` handler can't be updated
mid-decode.

Two patterns that actually work:

**1. Cancel from inside the token callback** based on state the
callback can observe directly (elapsed time, accumulated content,
token budget beyond `opts.maxTokens`):

```js
const startMs = performance.now();
session.generate(opts, (toks) => {
    self.postMessage({ kind: 'tokens', toks });
    if (performance.now() - startMs > 30_000) {
        session.cancel();  // 30-second budget
    }
});
```

**2. `SharedArrayBuffer` + `Atomics`** for true cross-thread
signalling. Allocate an `Int32Array` on a `SharedArrayBuffer`,
poll it from inside the callback with `Atomics.load`, set it from
the main thread with `Atomics.store`. **Requires cross-origin
isolation** in browsers (`Cross-Origin-Opener-Policy: same-origin`
and `Cross-Origin-Embedder-Policy: require-corp` headers); transparent
on Node `worker_threads`.

```js
// main thread — only structured-cloneable data crosses the
// postMessage boundary. wasm-bindgen objects (GenerateOpts,
// Session, etc.) live inside wasm linear memory and would throw
// on postMessage; pass plain params and construct the object on
// the worker side.
const sab = new SharedArrayBuffer(4);
const cancelFlag = new Int32Array(sab);
worker.postMessage({ kind: 'init', cancelFlag });
worker.postMessage({ kind: 'generate', params: { maxTokens: 64 } });
// later, to cancel:
Atomics.store(cancelFlag, 0, 1);

// worker.js — `generate()` is invoked from inside the message
// handler so `cancelFlag` is guaranteed initialized first; running
// `generate` at top-level would race with `init` and crash on
// `Atomics.load(undefined, 0)`. The `cancelFlag` null check guards
// against a `generate` message arriving before `init` (out-of-order
// messages).
let cancelFlag;
self.onmessage = (ev) => {
    if (ev.data.kind === 'init') {
        cancelFlag = ev.data.cancelFlag;
    } else if (ev.data.kind === 'generate') {
        const opts = new GenerateOpts();
        Object.assign(opts, ev.data.params);  // setters fire per field
        session.generate(opts, (toks) => {
            self.postMessage({ kind: 'tokens', toks });
            if (cancelFlag && Atomics.load(cancelFlag, 0) !== 0) {
                session.cancel();
            }
        });
    }
};
```

A pure `postMessage`-flag pattern only works if the cancel arrives
before `generate` is called (the listener runs during the gap
between message receipt and the next `generate`). It's unreliable
for in-flight cancellation; use one of the two patterns above.

### Resuming after cancel vs starting over

After a cancellation lands (`finishReason === "Cancelled"` from
`generate`, or a thrown error from `appendText` / `appendTokens`),
two primitives let JS callers continue with the same `Session`
without paying `engine.newSession(config)` setup cost again:

| API | KV cache | `position` | Sampler | When to use |
|---|---|---|---|---|
| `session.clearCancel()` | preserved | preserved | preserved | "interrupted but continuing" — keep the conversation context, append more tokens, generate again |
| `session.reset()` | dropped | reset to 0 | re-seeded from `cfg.seed` | "clear conversation" UI button — start fresh |

`clearCancel()` takes `&self`, `reset()` takes `&mut self` —
remember `reset()` must be invoked outside any in-flight `generate`
(wasm-bindgen's borrow check rejects re-entry).

```js
try {
    session.appendTokens(longPrompt);  // may throw "cancelled" mid-prefill
} catch (e) {
    if (String(e).includes('cancelled')) {
        session.clearCancel();           // resume without losing KV
        session.appendTokens(remainingTokens);
    } else {
        throw e;
    }
}
```

Bundlers without native wasm support need a loader plugin; see the
[`wasm-pack` bundler guide](https://rustwasm.github.io/docs/wasm-pack/tutorials/npm-browser-packages/getting-started.html)
for webpack / Rollup / Parcel specifics.

For no-bundler workflows (`<script type="module">` directly in the
browser, or `require('@hyeons-lab/cera-wasm')` from CommonJS Node),
download the `cera-wasm-pkg-web` or `cera-wasm-pkg-nodejs` artifact
instead — see the `Install` section above for the per-target
table.

## Building from source

```sh
just wasm        # bundler target → cera-wasm/pkg-bundler/
just wasm-web    # browser ESM    → cera-wasm/pkg-web/
just wasm-node   # CommonJS Node  → cera-wasm/pkg-nodejs/
```

Requires `wasm-pack` (`cargo install wasm-pack`) and `wasm-opt`
(macOS: `brew install binaryen`; linux: `apt-get install binaryen`).

## Multi-threaded build

Threaded variants light up `cera`'s rayon paths (batched prefill
GEMM, parallel GEMV row sweeps, dequant_rows_to_f32) on wasm via
`wasm-bindgen-rayon`. The generated package surfaces an
`initThreadPool(numThreads)` JS export that callers `await` once
before driving inference.

```sh
just wasm-web-mt    # browser ESM + threads → cera-wasm/pkg-web-mt/
just wasm-node-mt   # CommonJS Node + threads → cera-wasm/pkg-nodejs-mt/
```

Both recipes set the atomics target-feature, the
shared-memory/import-memory link args, and `-Z build-std=panic_abort,std`
together. Single-threaded `wasm`/`wasm-web`/`wasm-node` recipes are
unchanged. `--target bundler` is intentionally not provided —
`wasm-bindgen-rayon` doesn't have canonical bundler-side worker glue.

Extra prerequisite (single-threaded builds don't need this): the
`rust-src` rustup component for `-Z build-std`:

```sh
rustup component add rust-src
```

### Browser usage

```js
import init, { initThreadPool, CeraEngine } from './cera_wasm.js';

await init();
await initThreadPool(navigator.hardwareConcurrency);
// ...drive inference normally; rayon paths now run on the worker pool
```

The host page **must** be served with cross-origin isolation headers
so the browser hands out a real `SharedArrayBuffer`:

```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
```

Without these `initThreadPool` rejects with a `SharedArrayBuffer is
not defined` error. Most static-file dev servers don't set them by
default — see the `wasm-bindgen-rayon` README for snippets covering
Vite, webpack-dev-server, `http-server`, and similar tools.

### Node usage

`pkg-nodejs-mt/` ships as a CommonJS module, so top-level `await`
isn't available — wrap the init in an async IIFE (or run the
snippet from a `.mjs` / `"type": "module"` ESM file):

```js
const { initThreadPool, CeraEngine } = require('@hyeons-lab/cera-wasm');
const os = require('os');

(async () => {
    await initThreadPool(os.cpus().length);
    // ...drive inference normally; rayon paths now run on worker_threads
})();
```

Node has no equivalent of the browser's COOP/COEP gate — the
threaded build runs out-of-the-box on `worker_threads`.

## License

Apache-2.0 OR MIT, matching the rest of the cera workspace.
