New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — without the hours of GPU training that prior methods required.
Hackaday

Spectre And Meltdown: How Cache Works

The year so far has been filled with news of Spectre and Meltdown. These exploits take advantage of features like speculative execution, and memory access timing. What they have in common is the fact ...
GIGAZINE

An expert explains in an easy-to-understand way what CPU cache memory is

When talking about CPU specifications, in addition to clock speed and number of cores/threads, ' CPU cache memory ' is sometimes mentioned. Developer Gabriel G. Cunha explains what this CPU cache ...
EDN

Memory models for embedded multicore SoCs, Part 5 – Cache performance

Write-through: all cache memory writes are written to main memory, even if the data is retained in the cache, such as in the example in Figure 4.11. A cache line can be in two states – valid or ...
EDN

Memory Hierarchy Design – Part 1. Basics of Memory Hierarchies

Editor’s Note: Demand for increasing functionality and performance in systems designs continues to drive the need for more memory even as hardware engineers balance the dynamics of system capability, ...