#context
2 papers
-
inspiration
Prompt Compression Is Not KV Compression
Compressing the KV cache shrinks what the model stores. Prompt compression shrinks what you send. It cuts tokens before the model reads them, so cost, latency, and context bloat all fall.
-
inspiration
A Context Window Is Not Memory
A window is a buffer the model re-reads from scratch every call. Real memory is a deliberate design.