Describe the hierarchical cache structure and common cache misses.

• A. Caches at multiple levels (L1, L2, L3); Miss types: compulsory (first-time), capacity (not enough cache), conflict (due to mapping). Implications: performance depends on hit rate
• B. Cache misses: only compulsory
• C. Caches are only L1; Misses only due to capacity
• D. Cache always hits; misses do not occur

Answer: (A)

Caches are arranged in levels between the processor and main memory, with each level being larger and slower but cheaper than the next. The fastest, smallest level sits closest to the CPU and often has separate caches for instructions and data. The next level is bigger and a bit slower, and the next one after that is even larger and slower, frequently shared across cores. The idea is to keep the most frequently used data close to the processor so most accesses are quick; if something isn’t found in a given level, the search continues down the hierarchy until the data is found or main memory is reached.

Misses describe why a needed data block isn’t already in the cache. Compulsory misses happen on the first ever access to a data block because it hasn’t been loaded into any cache yet. Capacity misses occur when the working set is larger than the available cache space, so useful blocks must be evicted to make room for new ones even if they will be needed soon. Conflict misses arise from how blocks are mapped to a limited number of cache locations; with restricted associativity, multiple blocks may map to the same set and evict each other, causing misses even when some space is free elsewhere in the cache. In many designs, fully associative caches can avoid conflict misses, but practical CPUs balance associativity, area, and speed, so some conflict misses are common.

Overall performance depends on the hit rate: more hits in the upper levels mean faster memory access and fewer costly accesses to lower levels or main memory.