# BSV KVStore Module
> Blockchain-backed key-value storage for BSV applications

## Overview

This module provides persistent key-value storage backed by blockchain transactions. It offers two implementations:

- **LocalKVStore**: Private storage using wallet transactions and baskets. Values are encrypted by default using wallet-derived keys.
- **GlobalKVStore**: Public storage using the overlay network. Values are stored as PushDrop tokens discoverable via lookup services.

Both implementations maintain cross-SDK compatibility with the TypeScript and Go BSV SDKs.

## Files

| File | Purpose | Lines |
|------|---------|-------|
| `mod.rs` | Module root with re-exports and documentation | ~120 |
| `types.rs` | Core types (Config, Entry, Token, Query, Options, KvProtocolFields, TTL encoding) | ~849 |
| `local.rs` | LocalKVStore implementation with MockWallet tests | ~1743 |
| `global.rs` | GlobalKVStore implementation with SyncHistorian history building | ~826 |
| `interpreter.rs` | PushDrop token interpreter, KVStoreFields, signature verification | ~573 |

## Key Exports

```rust
// Types
pub use types::{
    KVStoreConfig, KVStoreEntry, KVStoreGetOptions, KVStoreLookupResult,
    KVStoreQuery, KVStoreRemoveOptions, KVStoreSetOptions, KVStoreToken,
};

// Stores
pub use local::LocalKVStore;
pub use global::GlobalKVStore;

// Interpreter
pub use interpreter::{KVStoreContext, KVStoreFields, KVStoreInterpreter};
```

## LocalKVStore

Private key-value store using wallet transactions and baskets.

### Features
- **Encrypted storage**: Values encrypted using wallet-derived keys (AES-256-GCM)
- **Wallet basket**: Entries stored in a namespaced basket (protocol_id)
- **Tag-based lookup**: Keys stored as tags for efficient retrieval
- **Atomic operations**: Per-key locking prevents concurrent conflicts
- **Validation**: Empty keys, values, and protocol_id are rejected
- **TTL support**: Optional time-to-live with client-side expiration
- **Batch operations**: `batch_get`, `batch_set`, `batch_remove` for multi-key operations

### Methods

```rust
impl<W: WalletInterface + std::fmt::Debug> LocalKVStore<W> {
    /// Create a new LocalKVStore (returns error if protocol_id is empty)
    pub fn new(wallet: W, config: KVStoreConfig) -> Result<Self>;

    /// Get a value by key (returns default_value if not found or TTL-expired)
    /// Matches Go SDK signature: Get(ctx, key, defaultValue string) (string, error)
    pub async fn get(&self, key: &str, default_value: &str) -> Result<String>;

    /// Get full entry with metadata (returns None if not found or TTL-expired)
    pub async fn get_entry(
        &self,
        key: &str,
        options: Option<KVStoreGetOptions>,
    ) -> Result<Option<KVStoreEntry>>;

    /// Set a key-value pair (creates or updates)
    pub async fn set(&self, key: &str, value: &str, options: Option<KVStoreSetOptions>) -> Result<String>;

    /// Remove a key-value pair
    pub async fn remove(&self, key: &str, options: Option<KVStoreRemoveOptions>) -> Result<Vec<String>>;

    /// List all keys
    pub async fn keys(&self) -> Result<Vec<String>>;

    /// List entries matching a query
    pub async fn list(&self, query: Option<KVStoreQuery>) -> Result<Vec<KVStoreEntry>>;

    /// Check if a key exists
    pub async fn has(&self, key: &str) -> Result<bool>;

    /// Get the entry count
    pub async fn count(&self) -> Result<usize>;

    /// Get multiple values by key (None for missing keys)
    pub async fn batch_get(&self, keys: &[&str]) -> Result<Vec<Option<String>>>;

    /// Set multiple key-value pairs sequentially
    pub async fn batch_set(&self, entries: &[(&str, &str)]) -> Result<()>;

    /// Remove multiple key-value pairs sequentially
    pub async fn batch_remove(&self, keys: &[&str]) -> Result<()>;

    /// Clear all entries
    pub async fn clear(&self) -> Result<()>;
}
```

### Encryption

LocalKVStore encrypts values using the wallet's key derivation:

```rust
// Key derivation for encryption
let protocol = Protocol::new(SecurityLevel::Counterparty, &config.protocol_id);
// keyID = the key being stored
// counterparty = Self_ (encrypt for self)
```

The encryption uses AES-256-GCM with keys derived via BRC-42.

## GlobalKVStore

Public key-value store using the overlay network.

### Features
- **Public discovery**: Entries discoverable via lookup services
- **PushDrop tokens**: Entries stored as on-chain tokens with field layout
- **Signature verification**: Entries signed by controller, verified on retrieval
- **Tag filtering**: Query by tags with "all" or "any" mode
- **Network presets**: Supports Mainnet and Testnet configurations
- **Cached identity key**: Controller pubkey cached via `Arc<Mutex<Option<String>>>`
- **History building**: Uses `SyncHistorian` to traverse transaction ancestry for entry history
- **Overlay broadcast**: Broadcasts tokens via `TopicBroadcaster`
- **Batch operations**: `batch_get`, `batch_set`, `batch_remove` for multi-key operations

### Methods

```rust
impl<W: WalletInterface> GlobalKVStore<W> {
    /// Create a new GlobalKVStore (uses default Mainnet network)
    pub fn new(wallet: W, config: KVStoreConfig) -> Self;

    /// Create with custom network preset
    pub fn with_network(wallet: W, config: KVStoreConfig, network: NetworkPreset) -> Self;

    /// Get a value by key
    pub async fn get(&self, key: &str, options: Option<KVStoreGetOptions>) -> Result<Option<KVStoreEntry>>;

    /// Set a key-value pair
    pub async fn set(&self, key: &str, value: &str, options: Option<KVStoreSetOptions>) -> Result<String>;

    /// Remove a key-value pair
    pub async fn remove(&self, key: &str, options: Option<KVStoreRemoveOptions>) -> Result<String>;

    /// Query entries
    pub async fn query(&self, query: KVStoreQuery) -> Result<Vec<KVStoreEntry>>;

    /// Get entries by controller
    pub async fn get_by_controller(&self, controller: &str) -> Result<Vec<KVStoreEntry>>;

    /// Get entries by tags
    pub async fn get_by_tags(&self, tags: &[String], mode: Option<&str>) -> Result<Vec<KVStoreEntry>>;

    /// Get multiple values by key (None for missing keys)
    pub async fn batch_get(&self, keys: &[&str]) -> Result<Vec<Option<KVStoreEntry>>>;

    /// Set multiple key-value pairs sequentially
    pub async fn batch_set(&self, entries: &[(&str, &str)]) -> Result<()>;

    /// Remove multiple key-value pairs sequentially
    pub async fn batch_remove(&self, keys: &[&str]) -> Result<()>;
}
```

## PushDrop Token Format

Global entries use PushDrop tokens with the following field layout:

| Index | Constant | Field | Description |
|-------|----------|-------|-------------|
| 0 | `PROTOCOL_ID` | Protocol ID | String identifier (e.g., "kvstore") |
| 1 | `KEY` | Key | The key string |
| 2 | `VALUE` | Value | The value (plaintext for global, encrypted for local) |
| 3 | `CONTROLLER` | Controller | 33-byte compressed public key |
| 4 | `TAGS` | Tags | JSON array of tag strings (optional, new format) |
| 5 | `SIGNATURE` | Signature | Controller's signature over fields 0-4 |

### Field Constants (KvProtocolFields)

```rust
pub struct KvProtocolFields;

impl KvProtocolFields {
    pub const PROTOCOL_ID: usize = 0;
    pub const KEY: usize = 1;
    pub const VALUE: usize = 2;
    pub const CONTROLLER: usize = 3;
    pub const TAGS: usize = 4;
    pub const SIGNATURE: usize = 5;
    pub const MIN_FIELDS_OLD: usize = 5;  // Old format without tags
    pub const MIN_FIELDS_NEW: usize = 6;  // New format with tags
}
```

### Old vs New Format

- **Old format**: 5 fields (no tags), signature at index 4
- **New format**: 6 fields (with tags), signature at index 5

Both formats are supported for backward compatibility.

## Configuration

```rust
pub struct KVStoreConfig {
    /// Protocol ID for key derivation (default: "kvstore")
    pub protocol_id: String,
    /// Service name for overlay lookup (default: "ls_kvstore")
    pub service_name: String,
    /// Token amount in satoshis (default: 1)
    pub token_amount: u64,
    /// Topics for broadcasting (default: ["tm_kvstore"])
    pub topics: Vec<String>,
    /// Originator for wallet operations
    pub originator: Option<String>,
    /// Whether to encrypt values (LocalKVStore only, default: true)
    pub encrypt: bool,
}

impl KVStoreConfig {
    pub fn new() -> Self;
    pub fn with_protocol_id(self, id: impl Into<String>) -> Self;
    pub fn with_service_name(self, name: impl Into<String>) -> Self;
    pub fn with_token_amount(self, amount: u64) -> Self;
    pub fn with_topics(self, topics: Vec<String>) -> Self;
    pub fn with_originator(self, originator: impl Into<String>) -> Self;
    pub fn with_encrypt(self, encrypt: bool) -> Self;
}
```

## Query Parameters

```rust
pub struct KVStoreQuery {
    pub key: Option<String>,           // Exact key match
    pub controller: Option<String>,    // Controller pubkey (hex)
    pub protocol_id: Option<String>,   // Protocol ID filter
    pub tags: Option<Vec<String>>,     // Tag filter
    pub tag_query_mode: Option<String>, // "all" (default) or "any"
    pub limit: Option<u32>,            // Max results
    pub skip: Option<u32>,             // Pagination offset
    pub sort_order: Option<String>,    // "asc" or "desc"
}

impl KVStoreQuery {
    pub fn new() -> Self;
    pub fn with_key(self, key: impl Into<String>) -> Self;
    pub fn with_controller(self, controller: impl Into<String>) -> Self;
    pub fn with_protocol_id(self, protocol_id: impl Into<String>) -> Self;
    pub fn with_tags(self, tags: Vec<String>) -> Self;
    pub fn with_tag_query_mode(self, mode: impl Into<String>) -> Self;
    pub fn with_limit(self, limit: u32) -> Self;
    pub fn with_skip(self, skip: u32) -> Self;
    pub fn with_sort_order(self, order: impl Into<String>) -> Self;
    pub fn to_json(&self) -> serde_json::Value;
}
```

## TTL (Time-to-Live) Support

LocalKVStore supports optional TTL on entries via `KVStoreSetOptions::with_ttl()`. TTL is enforced client-side; on-chain data remains immutable.

### How It Works

1. When `ttl` is set, the value is wrapped in a JSON envelope: `{"v":"<value>","e":<unix_ts>}`
2. On `get()` / `get_entry()`, the envelope is decoded and the expiration timestamp is checked
3. Expired entries return the default value (for `get()`) or `None` (for `get_entry()`)

### Internal Functions

```rust
// Encode a value with expiration timestamp
pub(crate) fn encode_value_with_ttl(value: &str, ttl: Duration) -> String;

// Decode a potentially TTL-wrapped value, returns (value, is_expired)
pub(crate) fn decode_value_with_ttl(stored: &str) -> (String, bool);
```

### Usage

```rust
use std::time::Duration;
use bsv_rs::kvstore::{LocalKVStore, KVStoreConfig, KVStoreSetOptions};

let opts = KVStoreSetOptions::new()
    .with_ttl(Duration::from_secs(3600)); // Expires in 1 hour
store.set("session", "token_abc", Some(opts)).await?;

// After 1 hour, get() returns default_value
let val = store.get("session", "expired").await?;
```

## Operation Options

```rust
pub struct KVStoreSetOptions {
    pub protocol_id: Option<String>,     // Override protocol ID
    pub description: Option<String>,     // Operation description
    pub token_amount: Option<u64>,       // Token amount override
    pub tags: Option<Vec<String>>,       // Tags to attach
    pub ttl: Option<std::time::Duration>, // Time-to-live (client-side expiration)
}

pub struct KVStoreGetOptions {
    pub history: bool,                   // Include full history chain
    pub include_token: bool,             // Include token data in result
    pub service_name: Option<String>,    // Override service name
}

pub struct KVStoreRemoveOptions {
    pub protocol_id: Option<String>,     // Override protocol ID
    pub description: Option<String>,     // Operation description
}
```

## Interpreter

The `KVStoreInterpreter` extracts entries from PushDrop scripts. Signature verification
uses a cached `ProtoWallet::anyone()` via `OnceLock` for efficient repeated verification.

```rust
impl KVStoreInterpreter {
    /// Interpret a transaction output
    pub fn interpret(
        tx: &Transaction,
        output_index: u32,
        ctx: Option<&KVStoreContext>,
    ) -> Option<KVStoreEntry>;

    /// Interpret a locking script directly
    pub fn interpret_script(
        script: &LockingScript,
        ctx: Option<&KVStoreContext>,
    ) -> Option<KVStoreEntry>;

    /// Verify signature of a KVStore token
    /// Uses ProtoWallet::anyone() with SecurityLevel::App protocol
    /// Returns true if signature is valid, false if invalid/missing
    pub fn verify_signature(fields: &KVStoreFields, protocol_id: &str) -> bool;

    /// Extract just the value
    pub fn extract_value(script: &LockingScript) -> Option<String>;

    /// Check if script is a KVStore token
    pub fn is_kvstore_token(script: &LockingScript) -> bool;

    /// Extract all raw fields
    pub fn extract_fields(script: &LockingScript) -> Option<KVStoreFields>;
}
```

### KVStoreFields

Raw field data extracted from a KVStore token:

```rust
pub struct KVStoreFields {
    pub protocol_id: Vec<u8>,
    pub key: Vec<u8>,
    pub value: Vec<u8>,
    pub controller: Vec<u8>,
    pub tags: Option<Vec<u8>>,
    pub signature: Option<Vec<u8>>,
    pub locking_public_key: PublicKey,
}

impl KVStoreFields {
    pub fn protocol_id_string(&self) -> Option<String>;
    pub fn key_string(&self) -> Option<String>;
    pub fn value_string(&self) -> Option<String>;
    pub fn value_bytes(&self) -> &[u8];
    pub fn controller_hex(&self) -> String;
    pub fn tags_vec(&self) -> Vec<String>;
    pub fn has_tags(&self) -> bool;
}
```

## Usage Examples

### LocalKVStore Basic Usage

```rust
use bsv_rs::kvstore::{LocalKVStore, KVStoreConfig};
use bsv_rs::wallet::ProtoWallet;
use bsv_rs::primitives::PrivateKey;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create wallet and store
    let wallet = ProtoWallet::new(Some(PrivateKey::random()));
    let config = KVStoreConfig::default();
    let store = LocalKVStore::new(wallet, config)?;  // Note: returns Result

    // Set a value
    let outpoint = store.set("user:name", "Alice", None).await?;
    println!("Stored at: {}", outpoint);

    // Get a value with default
    let name = store.get("user:name", "Unknown").await?;
    println!("Name: {}", name);

    // Get full entry with metadata
    if let Some(entry) = store.get_entry("user:name", None).await? {
        println!("Entry: {} = {}", entry.key, entry.value);
    }

    // List all keys
    let keys = store.keys().await?;
    println!("Keys: {:?}", keys);

    // Remove a value
    store.remove("user:name", None).await?;

    Ok(())
}
```

### Batch Operations

```rust
use bsv_rs::kvstore::{LocalKVStore, KVStoreConfig};

// Batch set multiple values
let entries = vec![("key1", "value1"), ("key2", "value2"), ("key3", "value3")];
store.batch_set(&entries).await?;

// Batch get (returns Vec<Option<String>>)
let values = store.batch_get(&["key1", "key2", "missing"]).await?;
// values[0] = Some("value1"), values[2] = None

// Batch remove
store.batch_remove(&["key1", "key2"]).await?;
```

### TTL (Time-to-Live)

```rust
use std::time::Duration;
use bsv_rs::kvstore::KVStoreSetOptions;

// Set with 5-minute TTL
let opts = KVStoreSetOptions::new()
    .with_ttl(Duration::from_secs(300));
store.set("session:token", "abc123", Some(opts)).await?;

// Value is available immediately
let val = store.get("session:token", "").await?;
assert_eq!(val, "abc123");

// After 5 minutes, get() returns the default value
// let val = store.get("session:token", "expired").await?;
// assert_eq!(val, "expired");
```

### GlobalKVStore with Tags

```rust
use bsv_rs::kvstore::{GlobalKVStore, KVStoreConfig, KVStoreSetOptions, KVStoreQuery};
use bsv_rs::wallet::ProtoWallet;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let wallet = ProtoWallet::new(None);
    let config = KVStoreConfig::default();
    let store = GlobalKVStore::new(wallet, config);

    // Set with tags
    let opts = KVStoreSetOptions::new()
        .with_tags(vec!["important".to_string(), "user-data".to_string()]);
    store.set("profile:123", r#"{"name": "Bob"}"#, Some(opts)).await?;

    // Query by tags
    let query = KVStoreQuery::new()
        .with_tags(vec!["important".to_string()])
        .with_tag_query_mode("any")
        .with_limit(10);
    let entries = store.query(query).await?;

    for entry in entries {
        println!("{}: {} (tags: {:?})", entry.key, entry.value, entry.tags);
    }

    Ok(())
}
```

### Custom Configuration

```rust
use bsv_rs::kvstore::KVStoreConfig;

let config = KVStoreConfig::new()
    .with_protocol_id("my-app-storage")
    .with_service_name("ls_myapp")
    .with_topics(vec!["tm_myapp".to_string()])
    .with_token_amount(100)
    .with_originator("myapp.example.com")
    .with_encrypt(false);  // Disable encryption for local store
```

## Error Handling

The module uses specific error types:

```rust
use bsv_rs::Error;

// LocalKVStore creation can fail
match LocalKVStore::new(wallet, config) {
    Ok(store) => { /* use store */ }
    Err(Error::KvStoreEmptyContext) => println!("Protocol ID cannot be empty"),
    Err(e) => println!("Failed to create store: {}", e),
}

// Operations validate keys and values
match store.set("", "value", None).await {
    Err(Error::KvStoreInvalidKey) => println!("Key cannot be empty"),
    _ => {}
}

match store.set("key", "", None).await {
    Err(Error::KvStoreInvalidValue) => println!("Value cannot be empty"),
    _ => {}
}

// General error handling
match store.get("key", "default").await {
    Ok(value) => println!("Value: {}", value),
    Err(Error::KvStoreError(msg)) => println!("Store error: {}", msg),
    Err(Error::KvStoreKeyNotFound(key)) => println!("Key not found: {}", key),
    Err(Error::KvStoreCorruptedState(msg)) => println!("Corrupted: {}", msg),
    Err(e) => println!("Other error: {}", e),
}
```

## Internal Design

### Per-Key Locking

Both `LocalKVStore` and `GlobalKVStore` implement per-key atomic locking using `tokio::sync::oneshot` channels
to prevent concurrent modifications to the same key. Operations acquire a lock before mutating and release
it after completion.

### Signature Verification

`GlobalKVStore` verifies token signatures on retrieval. The verification:
1. Concatenates fields 0-4 (protocol_id + key + value + controller + optional tags)
2. Uses `SecurityLevel::App` protocol with key_id `"kvstore-token"`
3. Verifies via `ProtoWallet::anyone()` with `Counterparty::Other(controller_pubkey)`

### History Building

`GlobalKVStore` uses the overlay module's `SyncHistorian` to traverse transaction input ancestry
and extract all previous values for a key/protocol combination, returned in chronological order (oldest first).

### TTL Encoding

TTL values are stored as JSON envelopes wrapping the original value: `{"v":"<value>","e":<expiry_unix_ts>}`.
On read, `decode_value_with_ttl()` transparently checks expiration. Plain (non-TTL) values pass through unchanged.
Expiration is client-side only; on-chain data remains immutable.

## Feature Flag

Enable the kvstore feature in `Cargo.toml`:

```toml
[dependencies]
bsv-rs = { version = "0.3", features = ["kvstore"] }
```

The kvstore feature requires the `overlay` feature (which enables `wallet` and `tokio`).

## Cross-SDK Compatibility

This module maintains API compatibility with:
- [TypeScript SDK](https://github.com/bitcoin-sv/ts-sdk) - `LocalKVStore`, `GlobalKVStore`, query types
- [Go SDK](https://github.com/bitcoin-sv/go-sdk) - `LocalKVStore`, `KVStoreInterface`

### Type Mapping

| TypeScript | Rust |
|------------|------|
| `KVStoreConfig` | `KVStoreConfig` |
| `KVStoreEntry` | `KVStoreEntry` |
| `KVStoreToken` | `KVStoreToken` |
| `KVStoreQuery` | `KVStoreQuery` |
| `LocalKVStore` | `LocalKVStore<W>` |
| `GlobalKVStore` | `GlobalKVStore<W>` |
| `kvStoreInterpreter` | `KVStoreInterpreter` |

### Go SDK Method Compatibility

The `LocalKVStore::get()` method signature matches the Go SDK:
```go
// Go SDK
func (l *LocalKVStore) Get(ctx context.Context, key, defaultValue string) (string, error)
```
```rust
// Rust SDK
pub async fn get(&self, key: &str, default_value: &str) -> Result<String>
```

## Related Documentation

- `../wallet/CLAUDE.md` - Wallet module (WalletInterface, encryption)
- `../overlay/CLAUDE.md` - Overlay module (LookupResolver, TopicBroadcaster)
- `../script/templates/CLAUDE.md` - PushDrop template
- `../CLAUDE.md` - Root SDK documentation
