# File Format

Detailed specification of the SRX archive format.

---

## Overview

SRX uses a chunk-based format designed for:

- **Random Access** - Extract any chunk independently
- **Streaming** - Process files larger than available memory
- **Integrity** - Per-chunk authentication
- **Extensibility** - TLV-based metadata for future compatibility

---

## File Structure

```
┌─────────────────────────┐
│      Fixed Header       │  32 bytes
│    Magic: "SRX1"        │
│    Version, Flags       │
│    Metadata Length      │
│    Chunk Count          │
├─────────────────────────┤
│   TLV Metadata Section  │  Variable
│    Compression Algo     │
│    Encryption Algo      │
│    Original Filename    │
│    Argon2 Parameters    │
│    Salt                 │
├─────────────────────────┤
│    Chunk Index Table    │  Variable
│    Offset, Sizes        │
│    Nonces, Auth Tags    │
├─────────────────────────┤
│   Encrypted Data Chunks │  Variable
│    Chunk 0..N           │
├─────────────────────────┤
│    Integrity Footer     │  32 bytes
│    BLAKE3 Hash          │
└─────────────────────────┘
```

---

## Fixed Header (32 bytes)

The header is always at the beginning of the file and has a fixed size.

### Layout

| Offset | Size | Field | Description |
|--------|------|-------|-------------|
| 0 | 4 | Magic | `SRX1` (0x53, 0x52, 0x58, 0x31) |
| 4 | 2 | Version | Format version (currently `0x0001`) |
| 6 | 2 | Flags | Bit flags (see below) |
| 8 | 4 | Metadata Length | Length of TLV metadata section |
| 12 | 4 | Chunk Count | Number of chunks in archive |
| 16 | 8 | Original Size | Original file size in bytes |
| 24 | 8 | Compressed Size | Total compressed size in bytes |

### Flags

| Bit | Flag | Description |
|-----|------|-------------|
| 0 | `ENCRYPTED` | Archive is encrypted |
| 1 | `COMPRESSED` | Archive is compressed |
| 2-7 | Reserved | For future use |
| 8-15 | Compression Algo | 0=None, 1=Zstd, 2=LZ4 |

### Example Header

```
53 52 58 31  00 01  00 03  00 00 01 00
│  │  │  │   │  │   │  │   │        └─ Metadata length: 256 bytes
│  │  │  │   │  │   │  │   └────────── Chunk count: 3
│  │  │  │   │  │   │  └────────────── Flags: encrypted + compressed (zstd)
│  │  │  │   │  └────────────────────── Version: 1
│  └──┴──┴──┘   └────────────────────── Magic: "SRX1"
```

---

## TLV Metadata Section

The metadata section uses Type-Length-Value (TLV) encoding for extensibility.

### TLV Format

| Field | Size | Description |
|-------|------|-------------|
| Type | 2 bytes | Tag identifier |
| Length | 2 bytes | Length of value |
| Value | Variable | Tag data |

### Defined Tags

| Tag ID | Name | Description |
|--------|------|-------------|
| `0x0001` | `ORIGINAL_FILENAME` | Original filename (UTF-8) |
| `0x0002` | `COMPRESSION_ALGO` | Compression algorithm (1 byte) |
| `0x0003` | `COMPRESSION_LEVEL` | Compression level (1 byte) |
| `0x0004` | `ENCRYPTION_ALGO` | Encryption algorithm (1 byte) |
| `0x0005` | `ARGON2_SALT` | Argon2 salt (16 bytes) |
| `0x0006` | `ARGON2_MEMORY` | Memory cost in KiB (4 bytes) |
| `0x0007` | `ARGON2_ITERATIONS` | Time cost (4 bytes) |
| `0x0008` | `ARGON2_PARALLELISM` | Parallelism (4 bytes) |
| `0x0009` | `CREATION_TIME` | Unix timestamp (8 bytes) |
| `0x000A` | `CHUNK_SIZE` | Chunk size in bytes (4 bytes) |
| `0x000B`-`0x00FF` | Reserved | For future standard tags |
| `0x0100`-`0xFFFF` | User-defined | Custom application tags |

### Example Metadata

```
00 01  00 10  64 6F 63 75 6D 65 6E 74 2E 70 64 66 00 00 00 00
│     │      └─ Filename: "document.pdf" (padded to 16 bytes)
│     └──────── Length: 16 bytes
└────────────── Tag: ORIGINAL_FILENAME

00 02  00 01  01
│     │      └─ Algorithm: Zstd (1)
│     └──────── Length: 1 byte
└────────────── Tag: COMPRESSION_ALGO

00 05  00 10  [16 bytes of salt]
│     │      └─ Salt value
│     └──────── Length: 16 bytes
└────────────── Tag: ARGON2_SALT
```

---

## Chunk Index Table

The chunk index enables random access to any chunk.

### Index Entry Format

Each entry is 40 bytes:

| Offset | Size | Field | Description |
|--------|------|-------|-------------|
| 0 | 8 | Offset | Chunk offset in file |
| 8 | 4 | Original Size | Original chunk size |
| 12 | 4 | Compressed Size | Compressed chunk size |
| 16 | 24 | Nonce | XChaCha20 nonce |
| 40 | 16 | Auth Tag | Poly1305 authentication tag |

### Example Index

```
Entry 0:
  Offset:         0x00000100  (256)
  Original Size:  0x00040000  (262144 bytes)
  Compressed:     0x00018000  (98304 bytes)
  Nonce:          [24 bytes]
  Auth Tag:       [16 bytes]

Entry 1:
  Offset:         0x00018100  (98304 + 256)
  Original Size:  0x00040000  (262144 bytes)
  Compressed:     0x00020000  (131072 bytes)
  Nonce:          [24 bytes]
  Auth Tag:       [16 bytes]
```

---

## Encrypted Data Chunks

Each chunk is independently encrypted and authenticated.

### Chunk Processing

```
Original Data → Compress → Encrypt → Write
```

### Chunk Structure

```
┌─────────────────────────────────────┐
│        Encrypted Chunk Data         │
│         (XChaCha20-Poly1305)        │
└─────────────────────────────────────┘
```

### Encryption Details

| Component | Value |
|-----------|-------|
| Cipher | XChaCha20-Poly1305 |
| Nonce | 24 bytes, random per chunk |
| Auth Tag | 16 bytes (stored in index) |
| Key | 32 bytes from Argon2id |

### Chunk Size

- **Default**: 256 KiB (262,144 bytes)
- **Minimum**: 4 KiB (4,096 bytes)
- **Maximum**: 16 MiB (16,777,216 bytes)

---

## Integrity Footer (32 bytes)

The footer contains a BLAKE3 hash of the entire file (excluding the footer itself).

### Layout

| Offset | Size | Field | Description |
|--------|------|-------|-------------|
| 0 | 32 | BLAKE3 Hash | Hash of all preceding data |

### Verification

```rust
// Pseudocode
let file_data = &file[..file.len() - 32];
let stored_hash = &file[file.len() - 32..];
let computed_hash = blake3::hash(file_data);

assert_eq!(stored_hash, computed_hash.as_bytes());
```

---

## Complete Example

### Small Archive (1 chunk)

```
Offset  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F
        ─────────────────────────────────────────────────
0x0000  53 52 58 31 00 01 00 03 00 00 01 00 00 00 00 01  SRX1............
0x0010  00 00 00 00 00 00 10 00 00 00 00 00 00 00 08 00  ................
        │ Header (32 bytes)                               │
0x0020  00 01 00 10 64 6F 63 75 6D 65 6E 74 2E 74 78 74  ....document.txt
0x0030  00 00 00 00 00 00 00 00 00 02 00 01 01 00 05 00  ................
0x0040  10 [16 bytes salt]  00 06 00 04 00 00 10 00 00  ................
        │ TLV Metadata (256 bytes)                        │
0x0120  00 00 00 00 00 00 01 40 00 00 10 00 00 00 08 00  .....@.........
0x0130  [24 bytes nonce]  [16 bytes auth tag]           ................
        │ Chunk Index (40 bytes)                          │
0x0148  [Encrypted chunk data...]                        ................
        │ Data Chunk (variable)                           │
0x0948  [32 bytes BLAKE3 hash]                            ................
        │ Footer (32 bytes)                               │
```

---

## Versioning

### Current Version: 1

### Backward Compatibility

- Readers MUST accept archives with unknown TLV tags
- Readers MUST accept archives with extra data in reserved areas
- Writers SHOULD use the minimum version that supports required features

### Future Versions

- Version 2 may add: Multi-file support, incremental updates
- Version 3 may add: Different encryption algorithms, compression dictionaries

---

## Security Properties

### Encryption

- **Algorithm**: XChaCha20-Poly1305
- **Key Size**: 256 bits
- **Nonce Size**: 192 bits (allows random generation)
- **Authentication**: Poly1305 (128-bit tag)

### Key Derivation

- **Algorithm**: Argon2id
- **Memory**: 64 MiB (configurable)
- **Iterations**: 3 (configurable)
- **Parallelism**: 4 threads (configurable)
- **Salt**: 128 bits, random

### Integrity

- **Algorithm**: BLAKE3
- **Output**: 256 bits
- **Scope**: Entire file (excluding footer)

---

## Implementation Notes

### Reading an Archive

1. Read and validate header (magic, version)
2. Parse TLV metadata
3. Read chunk index table
4. Seek to desired chunk offset
5. Decrypt and decompress chunk

### Writing an Archive

1. Write header with placeholder values
2. Write TLV metadata
3. Process chunks: compress → encrypt → write
4. Build chunk index table
5. Write index table
6. Compute and write BLAKE3 footer
7. Update header with final values

---

## See Also

- [SPEC.md](../SPEC.md) - Formal specification
- [Security](Security) - Security model and best practices
- [Library API](Library-API) - Programmatic access