Langsmart Publishes Industry’s First p95 Semantic Cache Benchmarks for On-Premises AI Gateway, Challenges Market: “Show Me the p95”

Testing Confirms 10.2x Faster Response Times, Exceeding Cloud-Hosted Alternatives SAN JOSE, Calif.--(BUSINESS WIRE)--March 17, 2026-- NVIDIA GTC 202 ...
Ars Technica

cache size vs. transistor count? OR a challenge to my professor!!

so i got in this pissing match with my cs instructor. he was telling the class that there are four transistors per bit of L2 cache on any given cpu with on-die, full-speed cache (not actually the ...

New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...
TMCnet

Penguin Solutions Introduces Industry's First Production-Ready CXL-Based KV Cache Server

Accelerating memory-dependent AI processes, Penguin's MemoryAI KV cache server increases memory capacity by integrating 3 TB of DDR5 main memory and up to eight 1 TB CXL Add-in Cards (AICs). Penguin ...