DUCATI: High-performance Address Translation by Extending TLB Reach of GPU-accelerated Systems

Conventional on-chip TLB hierarchies are unable to fully cover the growing application working-set sizes.
To make things worse, Last-Level TLB (LLT) misses require multiple accesses to the page table even with
the use of page walk caches. Consequently, LLT misses incur long address translation latency and hurt
performance. This paper proposes two low-overhead hardware mechanisms for reducing the frequency and
penalty of on-die LLT misses. The first, Unified CAche and TLB (UCAT), enables the conventional on-die
Last-Level Cache (LLC) to store cache lines and TLB entries in a single unified structure and increases ondie
TLB capacity significantly. The second, DRAM-TLB, memoizes virtual to physical address translations
in DRAM and reduces LLT miss penalty when UCAT is unable to fully cover total application working-set.
DRAM-TLB serves as the next larger level in the TLB hierarchy that significantly increases TLB coverage
relative to on-chip TLBs. The combination of these two mechanisms, DUCATI, is an address translation
architecture that improves GPU performance by 81% (up to 4.5x) while requiring minimal changes to the
existing system design. We show that DUCATI is within 20%, 5%, and 2% the performance of a perfect LLT
system when using 4KB, 64KB, and 2MB pages respectively.