[English](#en) | [中文](#zh)

---

<a id="en"></a>

# kind2sql: Convert Serialized Data to SQL Values

A Rust library for efficiently converting a stream of serialized binary data into a sequence of SQL-compatible string values.

## Table of Contents

- [Introduction](#introduction)
- [Features](#features)
- [Usage](#usage)
- [Design](#design)
- [API Reference](#api-reference)
- [Tech Stack](#tech-stack)
- [Directory Structure](#directory-structure)
- [A Little Story](#a-little-story)

## Introduction

`kind2sql` provides a mechanism to map structured binary data to a format ready for SQL `INSERT` statements. It is designed for scenarios where performance is critical, such as bulk-importing logs or other serialized records from a byte stream directly into a database without intermediate parsing into high-level structs.

## Features

- **High Performance**: Operates directly on byte buffers (`Buf`), minimizing allocations and copying.
- **Type-Safe Schema**: Uses an enum (`Kind`) to define the "schema" of the binary data, preventing deserialization errors.
- **Extensible**: Supports different SQL dialects through the `ToSql` trait. A MySQL implementation is provided out-of-the-box.
- **Efficient Encoding**: Uses variable-byte encoding for string and byte array lengths, saving space.
- **Feature Gated**: Core functionality is split into features (`to_sql`, `mysql`) to keep the dependency tree minimal.

## Usage

First, add `kind2sql` to your `Cargo.toml` and enable the desired features. For example, to use the MySQL converter:

```toml
[dependencies]
kind2sql = { version = "0.1.0", features = ["mysql"] }
```

Next, define your data schema with `Kind` and pass your serialized data to the `to_sql` function.

```rust
use kind2sql::{Kind, ToSql, mysql::MysqlToSql};

// 1. Define the data schema.
let kinds = [Kind::U8, Kind::I16, Kind::String];

// 2. Prepare the serialized data buffer.
let mut data = vec![];
// U8: 123
data.push(123u8);
// I16: -456 (little endian)
data.extend_from_slice(&(-456i16).to_le_bytes());
// String: "hello" (length-prefixed)
data.push(5u8); // vbyte encoded length
data.extend_from_slice(b"hello");

// 3. Convert the data.
let result = MysqlToSql::to_sql(&kinds, &data[..]).unwrap();

assert_eq!(result, vec!["123", "-456", "'hello'"]);
```

The output `Vec<String>` contains values that are properly formatted and escaped for a MySQL `INSERT` statement.

## Design

The library's design revolves around three main components:

1.  **`Kind` Enum**: This enum acts as a schema descriptor. An array of `Kind` variants defines the sequence and type of data fields packed in the binary buffer.

2.  **`to_sql` function**: This internal function is the engine of the library. It reads from a type implementing `bytes::Buf` and, based on the `Kind` provided, deserializes one value, converts it to a string, and advances the buffer. For `String` and `Bytes`, it first decodes a variable-byte integer to determine the length of the upcoming data.

3.  **`ToSql` Trait**: This trait abstracts the dialect-specific formatting, particularly for binary data (`BLOB`). To support a new database (e.g., PostgreSQL), you would implement `ToSql` and provide a `blob` function that formats byte arrays according to that database's requirements.

The overall process is:
`ToSql::to_sql` -> loops through `&[Kind]` -> calls internal `to_sql` for each `Kind` -> `to_sql` reads from `Buf`, formats data, and pushes to output vector.

## API Reference

### `enum Kind`

Defines the supported data types that can be deserialized from the byte buffer.

- `U8`, `I8`: 1-byte integers
- `U16`, `I16`: 2-byte little-endian integers
- `U32`: Variable-byte encoded unsigned 32-bit integer (1-5 bytes)
- `I32`: 4-byte little-endian signed integer
- `U64`: Variable-byte encoded unsigned 64-bit integer (1-10 bytes)
- `I64`: 8-byte little-endian signed integer
- `String`: A UTF-8 string, prefixed with its `vbyte`-encoded length
- `Bytes`: A byte array, prefixed with its `vbyte`-encoded length

### `trait ToSql`

A trait to be implemented by dialect-specific converters.

- `fn blob(data: &[u8]) -> String`: A required function that defines how a byte slice should be formatted for the target SQL dialect (e.g., `X'AABBCC'` for MySQL).
- `fn to_sql(...)`: The primary method that orchestrates the conversion process using the provided kinds and buffer.

### `mysql::MysqlToSql`

A concrete implementation of `ToSql` for MySQL, available under the `mysql` feature flag.

## Tech Stack

- [Rust](https://www.rust-lang.org/)
- `bytes`: For efficient buffer manipulation.
- `num_enum`: For converting the `Kind` enum from/to integers.
- `vb`: For `vbyte` encoding/decoding of lengths.
- `sqle`: For SQL string and blob escaping.

## Directory Structure

```
.
├── Cargo.toml      # Package manifest
├── AGENTS.md       # Agent instructions
├── readme/         # Documentation
│   ├── en.md       # English README
│   └── zh.md       # Chinese README
├── src/
│   ├── lib.rs      # Main library file, exports modules and types
│   └── to_sql.rs   # Core conversion logic and `ToSql` trait
└── tests/
    └── main.rs     # Integration tests
```

## A Little Story

The term "BLOB" (Binary Large Object) was coined by Jim Starkey at Digital Equipment Corporation (DEC) in the 1980s. According to Starkey, he had just watched the 1958 horror film *The Blob*, which features a gelatinous, amorphous alien that consumes everything in its path. He thought it was a fitting name for the data type he was working on, which was designed to store large, unstructured chunks of binary data in a database. The name stuck and is now a standard part of SQL, reminding us that even in the structured world of databases, there's a place for a bit of amorphous creativity.

---

## About

This project is an open-source component of [js0.site ⋅ Refactoring the Internet Plan](https://js0.site).

We are redefining the development paradigm of the Internet in a componentized way. Welcome to follow us:

* [Google Group](https://groups.google.com/g/js0-site)
* [js0site.bsky.social](https://bsky.app/profile/js0site.bsky.social)

---

<a id="zh"></a>

# kind2sql: 将序列化数据转换为SQL值

一个Rust库，用于将序列化二进制数据流高效转换为SQL兼容的字符串值序列。

## 目录

- [简介](#简介)
- [功能](#功能)
- [用法](#用法)
- [设计思路](#设计思路)
- [API参考](#api参考)
- [技术栈](#技术栈)
- [目录结构](#目录结构)
- [技术小故事](#技术小故事)

## 简介

`kind2sql` 提供一种将结构化二进制数据映射为可直接用于 SQL `INSERT` 语句格式的机制。它专为性能敏感的场景设计，例如，将日志或其他序列化记录从字节流批量导入数据库，而无需先解析为高级结构体。

## 功能

- **高性能**: 直接操作字节缓冲区 (`Buf`)，最大限度地减少内存分配和复制。
- **类型安全**: 使用 `Kind` 枚举来定义二进制数据的“模式”，防止反序列化错误。
- **可扩展**: 通过 `ToSql` trait 支持不同的 SQL 方言。默认提供 MySQL 实现。
- **高效编码**: 对字符串和字节数组的长度使用 `vbyte` 编码，节省空间。
- **功能门控**: 核心功能通过 Cargo features (`to_sql`, `mysql`) 分割，以保持依赖树最小化。

## 用法

首先，在 `Cargo.toml` 中添加 `kind2sql` 并启用所需功能。例如，使用 MySQL 转换器：

```toml
[dependencies]
kind2sql = { version = "0.1.0", features = ["mysql"] }
```

然后，使用 `Kind` 定义数据模式，并将序列化数据传递给 `to_sql` 函数。

```rust
use kind2sql::{Kind, ToSql, mysql::MysqlToSql};

// 1. 定义数据模式
let kinds = [Kind::U8, Kind::I16, Kind::String];

// 2. 准备序列化数据缓冲区
let mut data = vec![];
// U8: 123
data.push(123u8);
// I16: -456 (小端序)
data.extend_from_slice(&(-456i16).to_le_bytes());
// String: "hello" (长度前缀)
data.push(5u8); // vbyte 编码的长度
data.extend_from_slice(b"hello");

// 3. 转换数据
let result = MysqlToSql::to_sql(&kinds, &data[..]).unwrap();

assert_eq!(result, vec!["123", "-456", "'hello'"]);
```

输出的 `Vec<String>` 包含了为 MySQL `INSERT` 语句正确格式化和转义的值。

## 设计思路

该库的设计围绕三个核心组件：

1.  **`Kind` 枚举**: 此枚举充当模式描述符。`Kind` 变体的数组定义了二进制缓冲区中数据字段的序列和类型。

2.  **`to_sql` 函数**: 这个内部函数是库的引擎。它从实现 `bytes::Buf` 的类型中读取数据，并根据提供的 `Kind` 反序列化一个值，将其转换为字符串，然后移动缓冲区的游标。对于 `String` 和 `Bytes`，它首先解码一个 `vbyte` 整数以确定后续数据的长度。

3.  **`ToSql` Trait**: 此 trait 抽象了特定方言的格式化逻辑，特别是针对二进制数据（`BLOB`）。为了支持新的数据库（例如 PostgreSQL），你需要实现 `ToSql` 并提供一个 `blob` 函数，该函数根据该数据库的要求格式化字节数组。

整体调用流程如下：
`ToSql::to_sql` -> 遍历 `&[Kind]` -> 为每个 `Kind` 调用内部 `to_sql` 函数 -> `to_sql` 从 `Buf` 读取、格式化数据并推入输出向量。

## API参考

### `enum Kind`

定义可以从字节缓冲区反序列化的支持的数据类型。

- `U8`、`I8`: 1 字节整数
- `U16`、`I16`: 2 字节小端序整数
- `U32`: 变长字节编码的无符号 32 位整数（1-5 字节）
- `I32`: 4 字节小端序有符号整数
- `U64`: 变长字节编码的无符号 64 位整数（1-10 字节）
- `I64`: 8 字节小端序有符号整数
- `String`: UTF-8 字符串，其长度由前置的 `vbyte` 编码整数指定
- `Bytes`: 字节数组，其长度也由前置的 `vbyte` 编码整数指定

### `trait ToSql`

一个由特定方言转换器实现的 trait。

- `fn blob(data: &[u8]) -> String`: 必须实现的函数，定义字节切片应如何为目标 SQL 方言进行格式化（例如，MySQL 的 `X'AABBCC'`）。
- `fn to_sql(...)`: 库的主要方法，使用给定的 `kinds` 和 `buffer` 来编排整个转换过程。

### `mysql::MysqlToSql`

`ToSql` 的一个具体实现，用于 MySQL，在 `mysql` 功能标志下可用。

## 技术栈

- [Rust](https://www.rust-lang.org/)
- `bytes`: 用于高效的缓冲区操作。
- `num_enum`: 用于 `Kind` 枚举与整数之间的转换。
- `vb`: 用于 `vbyte` 编码/解码长度。
- `sqle`: 用于 SQL 字符串和 blob 的转义。

## 目录结构

```
.
├── Cargo.toml      # 包配置
├── AGENTS.md       # Agent 指示
├── readme/         # 文档
│   ├── en.md       # 英文 README
│   └── zh.md       # 中文 README
├── src/
│   ├── lib.rs      # 库主文件，导出模块和类型
│   └── to_sql.rs   # 核心转换逻辑和 `ToSql` trait
└── tests/
    └── main.rs     # 集成测试
```

## 技术小故事

术语 "BLOB"（二进制大对象）是 Jim Starkey 在 20 世纪 80 年代于数字设备公司（DEC）工作时创造的。据 Starkey 所述，他当时刚看完 1958 年的恐怖电影《The Blob》（一译《幽浮魔点》），影片中有一只无定形的凝胶状外星生物，吞噬其路径上的一切。他认为这个名字非常适合他正在开发的数据类型，该类型旨在数据库中存储非结构化的大块二进制数据。这个名字流传了下来，并成为 SQL 的标准部分，提醒着我们，即使在数据库这样结构化的世界里，也总有非晶体般创造力的空间。

---

## 关于

本项目为 [js0.site ⋅ 重构互联网计划](https://js0.site) 的开源组件。

我们正在以组件化的方式重新定义互联网的开发范式，欢迎关注：

* [谷歌邮件列表](https://groups.google.com/g/js0-site)
* [js0site.bsky.social](https://bsky.app/profile/js0site.bsky.social)
