spt-quant

Temperature-tiered vector quantization (f32/f16/u8/binary) for the SPT Format

Overview

Implements three quantization tiers mapped to access-temperature levels for vector data: scalar (int8, 4x compression) for hot data, product quantization (PQ, 8-16x) for warm data, and binary (1-bit, 32x) for cold data. A Count-Min Sketch tracks per-block access frequency to drive automatic promotion and demotion between tiers.

Key Features

• Scalar Quantization -- ScalarQuantizer compresses vectors to int8 (4x reduction)
• Product Quantization -- ProductQuantizer with configurable codebooks (8-16x reduction)
• Binary Quantization -- 1-bit encoding with hamming_distance for fast similarity (32x reduction)
• Temperature Tiering -- TemperatureTier drives hot/warm/cold placement decisions
• Access Sketching -- CountMinSketch for probabilistic access frequency tracking
• no_std Compatible -- Works without the standard library (disable std feature)

Dependencies

Key internal dependencies:

• spt-types -- Core SPT type definitions

Usage

use spt_quant::{
    ScalarQuantizer, ProductQuantizer, Quantizer,
    CountMinSketch, TemperatureTier,
    encode_binary, decode_binary, hamming_distance,
};

let sq = ScalarQuantizer::fit(&vectors);
let compressed = sq.encode(&vector);
let restored = sq.decode(&compressed);

API Reference

Core Types

Type	Description
Quantizer	Trait for encode/decode quantization
ScalarQuantizer	Int8 scalar quantization (hot tier)
ProductQuantizer	Product quantization with codebooks (warm tier)
TemperatureTier	Enum: Hot, Warm, Cold
CountMinSketch	Probabilistic access frequency tracker
encode_binary	Encode a vector to 1-bit binary
decode_binary	Decode a binary vector back to floats
hamming_distance	Fast similarity on binary-encoded vectors