Research Areas

High Performance Computing

Associated Publications

2023

Visibility Algorithms for Dynamic Dependence Analysis and Distributed Coherence
Michael Bauer, Elliott Slaughter, Sean Treichler, Wonchan Lee, Michael Garland, Alex Aiken
PPoPP
Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs
Benjamin Fuhrer, Yuval Shpigelman, Chen Tessler, Shie Mannor, Gal Chechik, Eitan Zahavy, Gal Dalal
CCGrid 2023

2022

Towards Precision-Aware Fault Tolerance Approaches for Mixed-Precision Applications
Bo Fang, Siva Hari, Timothy Tsai, Xinyi Li, Ganesh Gopalakrishnan, Ignacio Laguna, Kevin Barker, Ang Li
Workshop on Fault-Tolerance for HPC at Extreme Scale (FTXS)
Variable Bitrate Neural Fields
Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler
ACM SIGGRAPH 2022 Conference Proceedings
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
Thomas Müller, Alex Evans, Christoph Schied, Alex Keller
ACM Transactions on Graphics (SIGGRAPH 2022)
Best Technical Paper, SIGGRAPH 2022, THE BEST INVENTIONS OF 2022, TIME
Ray/Ribbon Intersections
Alexander Reshetov
Proc. ACM Comput. Graph. Interact. Tech., Vol. 5, No. 3, July 2022.
GATSPI: GPU Accelerated Gate-Level Simulation for Power Improvement
Yanqing Zhang, Mark Ren, Akshay Sridharan, Brucek Khailany
2022 Design Automation Conference

2021

GPS: A Global Publish-Subscribe Model for Multi-GPU Memory Management
Harini Muthukrishnan, Daniel Lustig, David Nellans, Thomas Wenisch
International Symposium on Microarchitecture (MICRO)
Best Paper Nominee, IEEE Micro Top Picks in Computer Architecture (Honorable Mention)
EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal in GPUs
Seung Won Min, Vikram Sharma Mailthody, Zaid Qureshi, Jinjun Xiong, Eiman Ebrahimi, Wen-mei Hwu
Proceedings of the VLDB Endownment (VLDB)
Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture
Seung Won Min, Kun Wu, Sitao Huang, Mert Hidayetoglu, Jinjun Xiong, Eiman Ebrahimi, Deming Chen, Wen-mei Hwu
Proceedings of the VLDB Endowment (VLDB)
Efficient Multi-GPU Shared Memory via Automatic Optimization of Fine-Grained Transfers
Harini Muthukrishnan, David Nellans, Daniel Lustig, Jeffrey Fessler, Thomas Wenisch
International Symposium on Computer Architecture (ISCA)
Demystifying GPU Reliability: Comparing and Combining Beam Experiments, Fault Simulation, and Profiling
Fernando Fernandes dos Santos, Siva Hari, Pedro Martins Basso, Luigi Carro, Paolo Rech
IEEE International Parallel & Distributed Processing Symposium (IPDPS)
Learning Sparse Matrix Row Permutations for Efficient SpMM on GPU Architectures
Atefeh Mehrabi, Donghyuk Lee, Niladrish Chatterjee, Danial J. Sorin, Benjamin C. Lee, Mike O'Connor
International Symposium on Performance Analysis of Systems and Software (ISPASS)
Scaling Implicit Parallelism via Dynamic Control Replication
Michael Bauer, Wonchan Lee, Elliott Slaughter, Zhihao Jia, Mario Di Renzo, Manolis Papadakis, Galen Shipman, Patrick McCormick, Michael Garland, Alex Aiken
Principles and Practices of Parallel Programming (PPoPP)

2020

Accelerating Reinforcement Learning through GPU Atari Emulation
Iuri Frosio, Steven Dalton
Advances in Neural Information Processing Systems 33 (NeurIPS 2020)
Locality-Centric Data and Threadblock Management for Massive GPUs
Mahmoud Khairy, Vadim Nikiforov, David Nellans, Timothy G. Rogers
International Symposium on Microarchitecture (MICRO)
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs
Esha Chouske, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Steve Keckler
International Symposium on Computer Architecture (ISCA)
An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives
Benjamin Klenk, Ted Jiang, Greg Thorson, Larry Dennison
International Symposium on Computer Architecture (ISCA)
NWChem: Past, Present, and Future
Edoardo Aprà, Many others, Oreste Villa, Many others
The Journal of Chemical Physics

2019

Near-Memory Data Transformation for Efficient Sparse Matrix Multi-Vector Multiplication
Daichi Fujiki, Niladrish Chatterjee, Donghyuk Lee, Mike O'Connor
International Conference for High-Performance Computing, Networking, Storage, a…
GPU-Accelerated Atari Emulation for Reinforcement Learning
Steven Dalton, Iuri Frosio, Michael Garland
Arxiv
GPU Snapshot: Checkpoint Offloading for GPU-Dense Systems
Kyushick Lee, Michael B. Sullivan, Siva Hari, Timothy Tsai, Steve Keckler, Mattan Erez
International Conference on Supercomputing
On the Trend of Resilience for GPU-Dense Systems
Kyushick Lee, Michael B. Sullivan, Siva Hari, Timothy Tsai, Steve Keckler, Mattan Erez
International Conference on Dependable Systems and Networks, Supplemental (DSN-…
NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation
Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, David Luebke
ACM Conference on Human-Computer-Interaction (CHI) 2019
A Fast and Robust Method for Avoiding Self-Intersection
Carsten Wächter, Nikolaus Binder
Ray Tracing Gems
Massively Parallel Path Space Filtering
Nikolaus Binder, Sascha Fricke, Alex Keller
arXiv
Metaoptimization on a Distributed System for Deep Reinforcement Learning
Greg Heinrich, Iuri Frosio
Massively Parallel Construction of Radix Tree Forests for the Efficient Sampling of Discrete Probability Distributions
Nikolaus Binder, Alex Keller
arXiv

2018

Exascale Deep Learning for Climate Analytics
Thorsten Kurth, Sean Treichler, Joshua Romero, Mayur Mudigonda, Nathan Luehr, Everett Phillips, Ankur Mahesh, Michael Matheson, Jack Deslippe, Massimiliano Fatica, Prabhat, Michael Houston
International Conference for High Performance Computing and Communications (SC'…
Dynamic Tracing: Memoization of Task Graphs for Dynamic Task-based Runtimes
Wonchan Lee, Elliott Slaughter, Michael Bauer, Sean Treichler, Todd Warszawski, Michael Garland, Alex Aiken
International Conference for High Performance Computing and Communications (SC'…
Evaluating and Accelerating High-Fidelity Error Injection for HPC
Chun-Kai Chang, Sangkug Lym, Nicholas Kelly, Michael B. Sullivan, Mattan Erez
The International Conference on High Performance Computing, Networking, Storage…
Exploiting Idle Resources in a High-Radix Switch for Supplemental Storage
Matthias Blumrich, Ted Jiang, Larry Dennison
Proceedings of the International Conference for High Performance Computing, Net…
Fast, High Precision Ray/Fiber Intersection using Tight, Disjoint Bounding Volumes
Nikolaus Binder, Alex Keller
arXiv
Massively Parallel Stackless Ray Tracing of Catmull-Clark Subdivision Surfaces
Nikolaus Binder, Alex Keller
arXiv
CRUM: Checkpoint-Restart Support for CUDA's Unified Memory
Rohan Garg, Apoorve Mohan, Michael B. Sullivan, Gene Cooperman
The International Conference on Cluster Computing (IEEE CLUSTER)
Phantom Ray-Hair Intersector
Alexander Reshetov, David Luebke
Proceedings of the ACM on Computer Graphics and Interactive Techniques
Hamartia: A Fast and Accurate Error Injection Framework
Chun-Kai Chang, Sangkug Lym, Nicholas Kelly, Michael B. Sullivan, Mattan Erez
The International Conference on Dependable Systems and Networks Workshops (DSN-…
Isometry: A Path-Based Distributed Data Transfer System
Zhihao Jia, Sean Treichler, Galen Shipman, Patrick McCormick, Alex Aiken
International Conference on Supercomputing (ICS)
BabelFlow: An Embedded Domain Specific Language for Parallel Analysis and Visualization
Steve Petruzza, Sean Treichler, Valerio Pascucci, Peer-Timo Bremer
International Parallel and Distributed Processing Symposium (IPDPS)
Scalable Collectives for Distributed Asynchronous Many-Task Runtimes
Matthew Whitlock, Hemanth Kolla, Sean Treichler, Philippe Pebay, Janine C. Bennett
International Parallel and Distributed Processing Symposium (IPDPS) - workshops

2017

Integrating External Resources with a Task-Based Programming Model
Zhihao Jia, Sean Treichler, Galen Shipman, Michael Bauer, Noah Watkins, Carlos Maltzahn, Patrick McCormick, Alex Aiken
International Conference on High Performance Computing (HiPC)
AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks
Aditya Devarakonda, Maxim Naumov, Michael Garland
arXiv:1712.02029 [cs.LG]
Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics
Liang Shi, Fu-Chung Huang, Ward Lopes, Wojciech Matusik, David Luebke
ACM SIGGRAPH ASIA 2017
Low Communication FMM-Accelerated FFT on GPUs
Cris Cecka
The International Conference for High Performance Computing, Networking, Storag…
Parallel Jaccard and Related Graph Clustering Techniques
Alexandre Fender, Nahid Emad, Serge Petiton, Joe Eaton, Maxim Naumov
Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for L…
Control Replication: Compiling Implicit Parallelism to Efficient SPMD with Logical Regions
Elliott Slaughter, Wonchan Lee, Sean Treichler, Wen Zhang, Michael Bauer, Galen Shipman, Patrick McCormick, Alex Aiken
 International Conference for High Performance Computing and Communications (SC…
A Novel Shard-Based Approach for Asynchronous Many-Task Models for In Situ Analysis
Philippe P. Pébaÿ, Giulio Borghesi, Hemanth Kolla, Janine C. Bennett, Sean Treichler
Workshop on In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visu…
Fine-Grained DRAM: Energy-Efficient DRAM for Extreme Bandwidth Systems
Mike O'Connor, Niladrish Chatterjee, Donghyuk Lee, John Wilson, Aditya Agrawal, Steve Keckler, William Dally
International Symposium on Microarchitecture (MICRO)
Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form
Maxim Naumov
arXiv:1709.06080 [cs.LG]
Exploiting Budan-Fourier and Vincent’s Theorems for Ray Tracing 3D Bézier Curves
Alexander Reshetov
High-Performance Graphics 2017
Parallel Modularity Clustering
Alexandre Fender, Nahid Emad, Serge Petiton, Maxim Naumov
Procedia Computer Science (Elsevier)
Relaxations for High-Performance Message Passing on Massively Parallel SIMT Processors
Benjamin Klenk, Holger Fröning, Hans Eberle, Larry Dennison
32nd IEEE International Parallel and Distributed Processing
Best Paper Award
The Iray Light Transport Simulation and Rendering System
Alex Keller, Carsten Wächter, Matthias Raab, Daniel Seibert, Dietger van Antwerpen, Johann Korndörfer, Lutz Kettner
arXiv
SASSIFI: An Architecture-level Fault Injection Tool for GPU Application Resilience Evaluation
Siva Hari, Timothy Tsai, Mark Stephenson, Steve Keckler, Joel Emer
International Symposium on Performance Analysis of Systems and Software (ISPASS)
Parallel Depth-First Search for Directed Acyclic Graphs
Maxim Naumov, Alysson Vrielink, Michael Garland
Technical Report NVR-2017-001

2016

Tensor Contractions with Extended BLAS Kernels on CPU and GPU
Yang Shi, U. N. Niranjan, Animashree Anandkumar, Cris Cecka
2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
Tensor Contractions with Extended BLAS Kernels on CPU and GPU
Yang Shi, U. N. Niranjan, Animashree Anandkumar, Cris Cecka
2016 IEEE 23rd International Conference on High Performance Computing (HiPC)
vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design.
Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, Steve Keckler
International Symposium on Microarchitecture (MICRO)
Approxilyzer: Towards A Systematic Framework for Instruction-Level Approximate Computing and its Application to Hardware Resiliency
Radha Venkatagiri, Abdulrahman Mahmoud, Siva Hari, Sarita Adve
International Symposium on Microarchitecture (MICRO)
All-Inclusive ECC: Thorough End-to-End Protection for Reliable Computer Memory
Jungrae Kim, Michael B. Sullivan, Sangkug Lym, Mattan Erez
The International Symposium on Computer Architecture (ISCA)
S-Step and Communication-Avoiding Iterative Methods
Maxim Naumov
Technical Report NVR-2016-003
Selective GPU Caches to Eliminate CPU-GPU HW Cache Coherence
Neha Agarwal, David Nellans, Eiman Ebrahimi, Thomas F. Wenisch, John Danskin, Steve Keckler
International Symposium on High Performance Computer Architecture (HPCA)
Towards High Performance Paged Memory for GPUs
Tianhao Zheng, David Nellans, Arslan Zulfiqar, Mark Stephenson, Steve Keckler
International Symposium on High Performance Computer Architecture (HPCA)
A Case for Toggle-Aware Compression for GPU Systems
Gennady Pekhimenko, Evgeny Bolotin, Nandita Vijaykumar, Onur Mutlu, Todd C. Mowry, Steve Keckler
International Symposium on High Performance Computer Architecture (HPCA)
Parallel Spectral Graph Partitioning
Maxim Naumov, Timothy Moon
Technical Report NVR-2016-001

2015

Network Endpoint Congestion Control for Fine-Grained Communication
Ted Jiang, Larry Dennison, William Dally
SC15
The Light Field Stereoscope
Fu-Chung Huang, David Luebke, Gordon Wetzstein
ACM SIGGRAPH 2015 Emerging Technologies
Parallel Graph Coloring with Applications to the Incomplete-LU Factorization on the GPU
Maxim Naumov, Patrice Castonguay, Jonathan Cohen
Technical Report NVR-2015-001
In-Memory Graph Databases for Web-Scale Data
Vito Giovanni Castellana, Alessandro Morari, Jesse Weaver, Antonino Time, David Haglin, Oreste Villa, John Feo
IEEE Computer

2014

Scaling the Power Wall: A Path to Exascale
Oreste Villa, Daniel Johnson, Mike O'Connor, Evgeny Bolotin, David Nellans, Justin Luitjens, Nikolai Sakharnykh, Peng Wang, Paulius Micikevicius, Anthony Scudiero, Steve Keckler, William Dally
SC '14

2012

Preconditioned Block-Iterative Methods on GPUs
Maxim Naumov
Proceedings in Applied Mathematics and Mechanics
Incomplete-LU and Cholesky Factorization in the Preconditioned Iterative Methods on the GPU
Maxim Naumov
Technical Report NVR-2012-003
Efficient Parallel Merge Sort for Fixed and Variable Length Keys
Andrew Davidson, David Tarjan, Michael Garland, John Owens
Proc. Innovative Parallel Computing
Scalable GPU Graph Traversal
Duane Merrill, Michael Garland, Andrew Grimshaw
17th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (…

2011

Allocation-oriented Algorithm Design with Application to GPU Computing, Ph.D. Dissertation
Duane Merrill
Department of Computer Science, University of Virginia
Thrust: A Productivity-Oriented Library for CUDA
Nathan Bell, Jared Hoberock
GPU Computing Gems, Jade Edition, Edited by Wen-mei W. Hwu
High Performance and Scalable GPU Graph Traversal
Duane Merrill, Michael Garland, Andrew Grimshaw
Technical Report CS-2011-05, Department of Computer Science, University of Virg…
Parallel Solution of Sparse Triangular Linear Systems in the Preconditioned Iterative Methods on the GPU
Maxim Naumov
Technical Report NVR-2011-001

2010

Scalable Fluid Simulation using Anisotropic Turbulence Particles
Tobias Pfaff, Nils Thurey, Jonathan Cohen, Sarah Tariq, Markus Gross
ACM Transactions on Graphics (SIGGRAPH Asia 2010)
Sparse Matrix-Vector Multiplication on Multicore and Accelerators
Sam Williams, Nathan Bell, Jee Whan Choi, Michael Garland, Leonid Oliker, Richard Vuduc
Scientific Computing on Multicore and Accelerators
Interactive Fluid-Particle Simulation using Translating Eulerian Grids
Jonathan Cohen, Sarah Tariq, Simon Green
Interactive 3D Graphics and Games (I3D) 2010

2009

Implementing Sparse Matrix-Vector Multiplication on Throughput-Oriented Processors
Nathan Bell, Michael Garland
Proc. Supercomputing '09
A Fast Double Precision CFD Code Using CUDA
Jonathan Cohen, M. Jeroen Molemaker
Proceedings of Parallel Computational Fluid Dynamics 2009

2008

Low Viscosity Flow Simulations for Animation
M .Jeroen Molemaker, Jonathan Cohen, Sanjit Patel, Jun-Yong Noh
Symposium on Computer Animation (SCA) 2008

2005

A Survey of General-Purpose Computation on Graphics Hardware
John D. Owens, David Luebke, Naga Govindaraju, Mark Harris, Jens Krüger, Aaron Lefohn, Timothy J. Purcell
Eurographics 2005, State of the Art Reports

Researchers

Aamer Jaleel
Benjamin Klenk
Charles Loop
Chris Wyman
Cris Cecka
David Nellans
Dennis Abts
Donghyuk Lee
Evgeny Bolotin
Guillermo Marcus
Hans Eberle
Haoyu Yang
Harini Muthukrishnan
Isaac Gelado
Iuri Frosio
Jaideep Pathak
Josef Spjut
Matthias Blumrich
Michael Garland
Michael Bauer
Mike O'Connor
Nicolai Oswald
Nikolaus Binder
Niladrish Chatterjee
Oreste Villa
Samuli Laine
Saurav Muralidharan
Sebastian Cammerer
Steve Keckler
Steven Dalton
Ted Jiang
Timothy Tsai
Vinu Joseph
Wen-mei Hwu
William Dally
Yaosheng Fu
Zander Majercik