Pure Rust port of NVIDIA's kvpress - KV cache compression for transformer models.
- Zero Python dependencies - pure Rust + Candle
- Multiple compression algorithms
- Trait-based design for easy extension
| Algorithm | Description | Use Case |
|---|---|---|
| KNormPress | L2 norm-based pruning | General purpose |
| SnapKVPress | Recent attention window | Streaming/chat |
| StreamingLLMPress | Attention sink preservation | Long sequences |
| ExpectedAttentionPress | Statistical prediction | Highest quality |
| RandomPress | Random selection | Baseline/testing |
use kvprs::{KNormPress, Press, CompressionContext, ModelConfig};
// Create a press with 50% compression
let press = KNormPress::new(0.5);
// Compress KV cache
let ctx = CompressionContext::new(keys, values, config);
let (compressed_keys, compressed_values) = press.compress(&ctx)?;[dependencies]
kvprs = { git = "https://github.com/..." } # Update with actual URLOr as a path dependency:
[dependencies]
kvprs = { path = "../kvprs", package = "kvpress-rs" }cargo testAll 6 tests passing.
Apache-2.0