Real-Time GPU Computing: Cache or No Cache?

Recent Graphics Processing Units (GPUs) have employed cache memories to boost performance. However, cache memories are well known to be harmful to time predictability for CPUs. For high-performance real-time systems using GPUs, it remains unknown whether or not cache memories should be employed. In this paper, we quantitatively compare the performance for GPUs with and without caches, and find that GPUs without the cache actually lead to better average-case performance, with higher time predictability. However, we also study a profiling-based cache bypassing method, which can use the L1 data cache more efficiently to achieve better average-case performance than that without the cache. Therefore, it seems still beneficial to employ caches for real-time computing on GPUs.