Distributed Shared-Memory Architectures Distributed shared-memory architectures Separate memory per processor Local or remote access via memory controller The physical address space is statically distributed Coherence Problems Simple approach: uncacheable shared data are marked as uncacheable and only private data are kept in caches very long latency to access memory for shared data Alternative: directory for memory blocks The directory per memory tracks state of every block in every cache which caches have a copies of the memory block, dirty vs. clean, ... Two – additional complications The interconnect cannot be used as a single point of arbitration like the bus Because the interconnect is message oriented, many messages must have explicit responses To prevent directory becoming the bottleneck, we distribute directory entries with memory, each keeping track of which processors have copies of their memory blocks Directory Protocols Similar to Snoopy Protocol: Three states 1 or more processors have the block cached, and the value in memory is Shared: up-to-date (as well as in all the caches) no processor has a copy of the cache block (not valid in any cache) Uncached: Exactly one processor has a copy of the cache block, and it has written Exclusive: the block, so the memory copy is out of date The processor is called the owner of the block
Distributed Shared-Memory Architectures Distributed shared-memory architectures Separate memory per processor Local or remote access via memory controller The physical address space is statically distributed Coherence Problems Simple approach: uncacheable shared data are marked as uncacheable and only private data are kept in caches very long latency to access memory for shared data Alternative: directory for memory blocks The directory per memory tracks state of every block in every cache which caches have a copies of the memory block, dirty vs. clean, ... – Two additional complications The interconnect cannot be used as a single point of arbitration like the bus Because the interconnect is message oriented, many messages must have explicit responses To prevent directory becoming the bottleneck, we distribute directory entries with memory, each keeping track of which processors have copies of their memory blocks Directory Protocols Similar to Snoopy Protocol: Three states Shared: 1 or more processors have the block cached, and the value in memory is up-to-date (as well as in all the caches) Uncached: no processor has a copy of the cache block (not valid in any cache) Exclusive: Exactly one processor has a copy of the cache block, and it has written the block, so the memory copy is out of date The processor is called the owner of the block