What is memory locality and why does the memory hierarchy matter for CPU performance?

• A. Locality reduces network latency
• B. Locality is not important with modern CPUs
• C. Temporal and spatial locality; caches exploit locality; memory hierarchy reduces average access time; misses and bandwidth impact performance
• D. Locality only affects spinning disks

Answer: (C)

Memory locality is the tendency of a program to reuse data that has been accessed recently (temporal locality) and to access data near those locations (spatial locality). This is why caches exist: they keep small, fast copies of data close to the CPU so that nearby or recently used data can be retrieved much quicker than from main memory.

The memory hierarchy is built to exploit this behavior. L1, L2, and L3 caches are fast, small storage near the CPU; main memory is larger but slower; disk storage is much slower still. When data is found in a cache (a cache hit), access is fast. When data isn’t in the upper levels (a cache miss), the processor must fetch from a slower level, incurring latency and using more bandwidth. The overall CPU performance depends on the cache hit rate and how often data has to be brought down the hierarchy.

That idea—temporal and spatial locality, caches exploiting locality, and the memory hierarchy reducing average access time with misses and bandwidth considerations—best captures why memory locality matters for CPU performance.

Choices that talk about network latency, claim locality isn’t important, or say locality only affects spinning disks don’t fit, because the core point is how locality enables fast cache access and how the hierarchy manages memory delays.