Research
Cache engineering paper proposes letting agents directly control their KV cache
The paper proposes drop and repos operations so agents can manage active KV cache without re-prefilling. A tool-response rolling-drop policy reportedly improves benchmark scores by 1.7 points and cuts average cost by 27.7%.
Every outlet that ran it 1 article
Hugging Face PapersResearchLead