Introduction to Processor Cache

What is Processor Cache?
Processor cache memory is the usually small amount of high-speed Static RAM (SRAM) that can significantly improve CPU performance. This cache resides between the CPU and the main system memory. There are two levels of processor cache: "Primary" and "Secondary." Primary cache, also known as Level 1 or L1, is that cache memory built into the processor itself and is usually very small, 4KB to 16KB. Secondary cache, or Level 2 (L2), is external from the CPU and can consist of different chip or module styles, but all usually operate at 12ns to 25ns.

Motherboard designers often make the secondary cache upgradable from as little as 32KB, all the way to 1MB and beyond. The technique used to access this cache determines the number of chips used to upgrade the cache, whether the upgrade is a module or individual chips. As with RAM, cache memory is addressed in 8 bit increments, which means 4 or 8 chip configurations are most common. Although cache rarely uses parity, additional chips are often used. These additional chips are called "TAG RAM" or "Dirty Bit" RAM. They help the cache controller keep track of where data is located within the cache, and for delaying the transfer of data from cache back to main RAM.

There are two key styles of cache: Write-Back (W-B) and Write-Through (W-T). Both styles serve the cache's primary purpose of allowing the CPU to retrieve data from cache when it's available there. This reduces the need to access the much slower main RAM. Sending data to memory is where W-B differs from W-T. Write-Through means data is sent, or written, to cache and main memory simultaneously. Write-Back cache allows the CPU to send data only to the cache, further reducing the need for the CPU to access slower main memory. The cache controller performs the maintenance of main memory, freeing the CPU from this task, thus allowing the CPU to return to more processor intensive operations. Write-Back cache is usually more expensive and more complex to design and build, but offers performance advantages over Write-Through.

Why isn't cache much larger?
On most systems, performance improves dramatically up with cache up to 256KB, but the performance benefit quickly flattens out after 256KB and offers almost no additional benefits after 512KB. The reason for this is that the larger the cache, the more difficult it is to keep track the data in cache and the longer it takes to determine the location and availability of data. Checking for data in a small cache takes very little time, but as the cache size increases, the time to check for data increases dramatically while the likelihood of the data being found increases only marginally.