SIP: Speculative Insertion Policy for High Performance Caching
High performance cache mechanisms have a great impact on overall performance of computer systems by reducing memory access latency. Least Recently Used LRU) mechanism can achieve good performance in small workload; however, it suffers from thrashing caused by memory intensive application. To address this challenge, dynamic insertion policy DIP, which dynamically switches between LRU and an alternative policy, has recently been proposed. The algo rithm, however, applies either one of the two policies to the entire cache based on the total number of misses. Therefore, such algorithm is not flexible enough to adjust to the different memory access pattern of each set. In this paper, we propose a novel approach, called SIP (Speculative Insertion Policy) to achieve high performa nce caching viaworkload speculation mechanism. By utilizing memory access direction and cache misses/hits SIP estimates the size of per set workload and dynamically selects an optimized policy for each set, leading to significant improvement in cache performance with only 13Kbits additional storage requirement over LRU for 1MB cache.