High Performance Computing

Associated Publications

2025

Task-Based Tensor Computations on Modern GPUs

Rohan Yadav, Michael Garland, Alex Aiken, Michael Bauer

Beyond the Buzz: A Pragmatic Take on Inference Disaggregation

Tiyasa Mitra, Ritika Borkar, Nidhi Bhatia, Ramon Matas, Shivam Raj, Dheevatsa Mudigere, Ritchie Zhao, Maximilian Golub, Arpan Dutta, Sailaja Madduri, Dharmesh Jani, Brian Pharris, Bita Darvish Rouhani

Arxiv

SLIM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression

Mohammad Mozaffari , Amir Yazdanbakhsh, Maryam Mehri Dehnavi

ICML 2025

Adaptive Algebraic Reuse of Reordering in Cholesky Factorizations with Dynamic Sparsity Patterns

Behrooz Zarebavani, Danny Kaufman, David Levin, Maryam Mehri Dehnavi

SIGGRAPH 2025

Automatic Tracing in Task-Based Runtime Systems

Rohan Yadav, Michael Bauer, David Broman, Michael Garland, Alex Aiken, Fredrik Kjolstad

ASPLOS

Composing Distributed Computations Through Task and Kernel Fusion

Rohan Yadav, Shiv Sundrum, Wonchan Lee, Michael Garland, Michael Bauer, Alex Aiken, Fredrik Kjolstad

ASPLOS

2024

Differentiable GPU-Parallelized Task and Motion Planning

William Shen, Caelan Garrett, Ankit Goyal, Tucker Hermans, Fabio Ramos

CoRL 2024 Workshop on Differentiable Optimization

2023

Legate Sparse: Distributed Sparse Computing in Python

Rohan Yadav, Wonchan Lee, Melih Elibol, Taylor Patti, Manolis Papadakis, Michael Garland, Alex Aiken, Fredrik Kjolstad, Michael Bauer

Supercomputing

Neuralangelo: High-Fidelity Neural Surface Reconstruction

Max Zhaoshuo Li, Thomas Müller, Alex Evans, Russell H. Taylor, Mathias Unberath, Ming-Yu Liu, Chen-Hsuan Lin

CVPR 2023

The Best Inventions of 2023, TIME Magazine

Visibility Algorithms for Dynamic Dependence Analysis and Distributed Coherence

Michael Bauer, Elliott Slaughter, Sean Treichler, Wonchan Lee, Michael Garland, Alex Aiken

PPoPP

Parsimony: Enabling SIMD/Vector Programming in Standard Compiler Flows

Vijay Kandiah, Daniel Lustig, Oreste Villa, David Nellans, Nikos Hardavellas

International Symposium on Code Generation and Optimization

Implementing Reinforcement Learning Datacenter Congestion Control in NVIDIA NICs

Benjamin Fuhrer, Yuval Shpigelman, Chen Tessler, Shie Mannor, Gal Chechik, Eitan Zahavy, Gal Dalal

CCGrid 2023

2022

Towards Precision-Aware Fault Tolerance Approaches for Mixed-Precision Applications

Bo Fang, Siva Hari, Timothy Tsai, Xinyi Li, Ganesh Gopalakrishnan, Ignacio Laguna, Kevin Barker, Ang Li

Workshop on Fault-Tolerance for HPC at Extreme Scale (FTXS)

Variable Bitrate Neural Fields

Towaki Takikawa, Alex Evans, Jonathan Tremblay, Thomas Müller, Morgan McGuire, Alec Jacobson, Sanja Fidler

ACM SIGGRAPH 2022 Conference Proceedings

Instant Neural Graphics Primitives with a Multiresolution Hash Encoding

Thomas Müller, Alex Evans, Christoph Schied, Alex Keller

ACM Transactions on Graphics (SIGGRAPH 2022)

Best Technical Paper, SIGGRAPH 2022, THE BEST INVENTIONS OF 2022, TIME

Ray/Ribbon Intersections

Alexander Reshetov

Proc. ACM Comput. Graph. Interact. Tech., Vol. 5, No. 3, July 2022.

GATSPI: GPU Accelerated Gate-Level Simulation for Power Improvement

Yanqing Zhang, Haoxing (Mark) Ren, Akshay Sridharan, Brucek Khailany

2022 Design Automation Conference

2021

GPS: A Global Publish-Subscribe Model for Multi-GPU Memory Management

Harini Muthukrishnan, Daniel Lustig, David Nellans, Thomas Wenisch

International Symposium on Microarchitecture (MICRO)

Best Paper nominee, IEEE Micro Top Picks in Computer Architecture (Honorable Mention)

EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal in GPUs

Seung Won Min, Vikram Sharma Mailthody, Zaid Qureshi, Jinjun Xiong, Eiman Ebrahimi, Wen-mei Hwu

Proceedings of the VLDB Endownment (VLDB)

Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture

Seung Won Min, Kun Wu, Sitao Huang, Mert Hidayetoglu, Jinjun Xiong, Eiman Ebrahimi, Deming Chen, Wen-mei Hwu

Proceedings of the VLDB Endowment (VLDB)

Suraksha: A Quantitative AV Safety Evaluation Framework to Analyze Safety Implications of Perception Design Choices

Hengyu Zhao, Siva Hari, Timothy Tsai, Michael B. Sullivan, Steve Keckler, Jishen Zhao

Workshop on Safety and Security of Intelligent Vehicles (SSIV)

Efficient Multi-GPU Shared Memory via Automatic Optimization of Fine-Grained Transfers

Harini Muthukrishnan, David Nellans, Daniel Lustig, Jeffrey Fessler, Thomas Wenisch

International Symposium on Computer Architecture (ISCA)

Demystifying GPU Reliability: Comparing and Combining Beam Experiments, Fault Simulation, and Profiling

Fernando Fernandes dos Santos, Siva Hari, Pedro Martins Basso, Luigi Carro, Paolo Rech

IEEE International Parallel & Distributed Processing Symposium (IPDPS)

Learning Sparse Matrix Row Permutations for Efficient SpMM on GPU Architectures

Atefeh Mehrabi, Donghyuk Lee, Niladrish Chatterjee, Danial J. Sorin, Benjamin C. Lee, Mike O'Connor

International Symposium on Performance Analysis of Systems and Software (ISPASS)

Large Graph Convolutional Network Training with GPU-Oriented Data Communication Architecture

Seung Won Min, Kun Wu, Sitao Huang, Mert Hidayetoglu, Jinjun Xiong, Eiman Ebrahimi, Deming Chen, Wen-mei Hwu

ArXiv

Scaling Implicit Parallelism via Dynamic Control Replication

Michael Bauer, Wonchan Lee, Elliott Slaughter, Zhihao Jia, Mario Di Renzo, Manolis Papadakis, Galen Shipman, Patrick McCormick, Michael Garland, Alex Aiken

Principles and Practices of Parallel Programming (PPoPP)

2020

Accelerating Reinforcement Learning through GPU Atari Emulation

Iuri Frosio, Steven Dalton

Advances in Neural Information Processing Systems 33 (NeurIPS 2020)

Locality-Centric Data and Threadblock Management for Massive GPUs

Mahmoud Khairy, Vadim Nikiforov, David Nellans, Timothy G. Rogers

International Symposium on Microarchitecture (MICRO)

EMOGI: Efficient Memory-access for Out-of-memory Graph-traversal In GPUs

Seung Won Min, Vikram Sharma Mailthody, Zaid Qureshi, Jinjun Xiong, Eiman Ebrahimi, Wen-mei Hwu

arXiv

Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs

Esha Chouske, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Steve Keckler

International Symposium on Computer Architecture (ISCA)

An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives

Benjamin Klenk, Ted Jiang, Greg Thorson, Larry Dennison

International Symposium on Computer Architecture (ISCA)

NWChem: Past, Present, and Future

Edoardo Aprà, Many others, Oreste Villa, Many others

The Journal of Chemical Physics

2019

Near-Memory Data Transformation for Efficient Sparse Matrix Multi-Vector Multiplication

Daichi Fujiki, Niladrish Chatterjee, Donghyuk Lee, Mike O'Connor

International Conference for High-Performance Computing, Networking, Storage, a…

Highly-scalable, Physics-informed GANs for Learning Solutions of Stochastic PDEs

Liu Yang, Sean Treichler, Thorsten Kurth, Keno Fischer, David Barajas-Solano, Josh Romero, Valentin Churavy, Alexandre Tartakovsky, Michael Houston, Prabhat, George Karniadakis

arXiv

Exascale Deep Learning for Scientific Inverse Problems

Nouamane Laanait, Joshua Romero, Junqi Yin, M. Todd Young, Sean Treichler, Vitalii Starchenko, Albina Borisevich, Alex Sergeev, Michael Matheson

arXiv

Task Bench: A Parameterized Benchmark for Evaluating Parallel Runtime Performance

Elliott Slaughter, Wei Wu, Yuankun Fu, Legend Brandenburg, Nicolai Garcia, Wilhem Kautz, Emily Marx, Kaleb S. Morris, Qinglei Cao, George Bosilca, Seema Mirchandaney, Wonchan Lee, Sean Treichler, Patrick McCormick, Alex Aiken

arXiv

Optimizing Multi-GPU Parallelization Strategies for Deep Learning Training

Saptadeep Pal, Eiman Ebrahimi, Arslan Zulfiqar, Yaosheng Fu, Victor Zhang, Szymon Migacz, David Nellans, Puneet Gupta

arXiv

GPU-Accelerated Atari Emulation for Reinforcement Learning

Steven Dalton, Iuri Frosio, Michael Garland

Arxiv

GPU Snapshot: Checkpoint Offloading for GPU-Dense Systems

Kyushick Lee, Michael B. Sullivan, Siva Hari, Timothy Tsai, Steve Keckler, Mattan Erez

International Conference on Supercomputing

On the Trend of Resilience for GPU-Dense Systems

Kyushick Lee, Michael B. Sullivan, Siva Hari, Timothy Tsai, Steve Keckler, Mattan Erez

International Conference on Dependable Systems and Networks, Supplemental (DSN-…

Best of SELSE (Workshop on Silicon Errors in Logic - System Effects)

NVGaze: An Anatomically-Informed Dataset for Low-Latency, Near-Eye Gaze Estimation

Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, David Luebke

ACM Conference on Human-Computer-Interaction (CHI) 2019

Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs

Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler

arXiv

DeLTA: GPU Performance Model for Deep Learning Applications with In-depth Memory System Traffic Analysis

Sangkug Lym, Donghyuk Lee, Mike O'Connor, Niladrish Chatterjee, Mattan Erez

arXiv

A Fast and Robust Method for Avoiding Self-Intersection

Carsten Wächter, Nikolaus Binder

Ray Tracing Gems

Massively Parallel Path Space Filtering

Nikolaus Binder, Sascha Fricke, Alex Keller

arXiv

Metaoptimization on a Distributed System for Deep Reinforcement Learning

Greg Heinrich, Iuri Frosio

Massively Parallel Construction of Radix Tree Forests for the Efficient Sampling of Discrete Probability Distributions

Nikolaus Binder, Alex Keller

arXiv

2018

Dynamic Tracing: Memoization of Task Graphs for Dynamic Task-based Runtimes

Wonchan Lee, Elliott Slaughter, Michael Bauer, Sean Treichler, Todd Warszawski, Michael Garland, Alex Aiken

International Conference for High Performance Computing and Communications (SC'…

Evaluating and Accelerating High-Fidelity Error Injection for HPC

Chun-Kai Chang, Sangkug Lym, Nicholas Kelly, Michael B. Sullivan, Mattan Erez

The International Conference on High Performance Computing, Networking, Storage…

Exascale Deep Learning for Climate Analytics

Thorsten Kurth, Sean Treichler, Joshua Romero, Mayur Mudigonda, Nathan Luehr, Everett Phillips, Ankur Mahesh, Michael Matheson, Jack Deslippe, Massimiliano Fatica, Prabhat, Michael Houston

International Conference for High Performance Computing and Communications (SC'…

Exploiting Idle Resources in a High-Radix Switch for Supplemental Storage

Matthias Blumrich, Ted Jiang, Larry Dennison

Proceedings of the International Conference for High Performance Computing, Net…

Fast, High Precision Ray/Fiber Intersection using Tight, Disjoint Bounding Volumes

Nikolaus Binder, Alex Keller

arXiv

Massively Parallel Stackless Ray Tracing of Catmull-Clark Subdivision Surfaces

Nikolaus Binder, Alex Keller

arXiv

Exascale Deep Learning for Climate Analytics

Thorsten Kurth, Sean Treichler, Joshua Romero, Mayur Mudigonda, Nathan Luehr, Everett Phillips, Ankur Mahesh, Michael Matheson, Jack Deslippe, Massimiliano Fatica, Prabhat, Michael Houston

arXiv

CRUM: Checkpoint-Restart Support for CUDA's Unified Memory

Rohan Garg, Apoorve Mohan, Michael B. Sullivan, Gene Cooperman

The International Conference on Cluster Computing (IEEE CLUSTER)

Phantom Ray-Hair Intersector

Alexander Reshetov, David Luebke

Proceedings of the ACM on Computer Graphics and Interactive Techniques

Hamartia: A Fast and Accurate Error Injection Framework

Chun-Kai Chang, Sangkug Lym, Nicholas Kelly, Michael B. Sullivan, Mattan Erez

The International Conference on Dependable Systems and Networks Workshops (DSN-…

Isometry: A Path-Based Distributed Data Transfer System

Zhihao Jia, Sean Treichler, Galen Shipman, Patrick McCormick, Alex Aiken

International Conference on Supercomputing (ICS)

Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training

Maohua Zhu, Jason Clemons, Jeff Pool, Minsoo Rhu, Steve Keckler, Yuan Xie

arXiv

Scalable Collectives for Distributed Asynchronous Many-Task Runtimes

Matthew Whitlock, Hemanth Kolla, Sean Treichler, Philippe Pebay, Janine C. Bennett

International Parallel and Distributed Processing Symposium (IPDPS) - workshops

BabelFlow: An Embedded Domain Specific Language for Parallel Analysis and Visualization

Steve Petruzza, Sean Treichler, Valerio Pascucci, Peer-Timo Bremer

International Parallel and Distributed Processing Symposium (IPDPS)

2017

Integrating External Resources with a Task-Based Programming Model

Zhihao Jia, Sean Treichler, Galen Shipman, Michael Bauer, Noah Watkins, Carlos Maltzahn, Patrick McCormick, Alex Aiken

International Conference on High Performance Computing (HiPC)

AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks

Aditya Devarakonda, Maxim Naumov, Michael Garland

arXiv:1712.02029 [cs.LG]

Near-eye Light Field Holographic Rendering with Spherical Waves for Wide Field of View Interactive 3D Computer Graphics

Liang Shi, Fu-Chung Huang, Ward Lopes, Wojciech Matusik, David Luebke

ACM SIGGRAPH ASIA 2017

A Novel Shard-Based Approach for Asynchronous Many-Task Models for In Situ Analysis

Philippe P. Pébaÿ, Giulio Borghesi, Hemanth Kolla, Janine C. Bennett, Sean Treichler

Workshop on In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visu…

Low Communication FMM-Accelerated FFT on GPUs

Cris Cecka

The International Conference for High Performance Computing, Networking, Storag…

Parallel Jaccard and Related Graph Clustering Techniques

Alexandre Fender, Nahid Emad, Serge Petiton, Joe Eaton, Maxim Naumov

Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for L…

Control Replication: Compiling Implicit Parallelism to Efficient SPMD with Logical Regions

Elliott Slaughter, Wonchan Lee, Sean Treichler, Wen Zhang, Michael Bauer, Galen Shipman, Patrick McCormick, Alex Aiken

International Conference for High Performance Computing and Communications (SC…

Fine-Grained DRAM: Energy-Efficient DRAM for Extreme Bandwidth Systems

Mike O'Connor, Niladrish Chatterjee, Donghyuk Lee, John Wilson, Aditya Agrawal, Steve Keckler, William Dally

International Symposium on Microarchitecture (MICRO)

Feedforward and Recurrent Neural Networks Backward Propagation and Hessian in Matrix Form

Maxim Naumov

arXiv:1709.06080 [cs.LG]

Exploiting Budan-Fourier and Vincent’s Theorems for Ray Tracing 3D Bézier Curves

Alexander Reshetov

High-Performance Graphics 2017

Parallel Modularity Clustering

Alexandre Fender, Nahid Emad, Serge Petiton, Maxim Naumov

Procedia Computer Science (Elsevier)

Relaxations for High-Performance Message Passing on Massively Parallel SIMT Processors

Benjamin Klenk, Holger Fröning, Hans Eberle, Larry Dennison

32nd IEEE International Parallel and Distributed Processing

Best Paper Award

The Iray Light Transport Simulation and Rendering System

Alex Keller, Carsten Wächter, Matthias Raab, Daniel Seibert, Dietger van Antwerpen, Johann Korndörfer, Lutz Kettner

arXiv

SASSIFI: An Architecture-level Fault Injection Tool for GPU Application Resilience Evaluation

Siva Hari, Timothy Tsai, Mark Stephenson, Steve Keckler, Joel Emer

International Symposium on Performance Analysis of Systems and Software (ISPASS)

Parallel Depth-First Search for Directed Acyclic Graphs

Maxim Naumov, Alysson Vrielink, Michael Garland

Technical Report NVR-2017-001

2016

Tensor Contractions with Extended BLAS Kernels on CPU and GPU

Yang Shi, U. N. Niranjan, Animashree Anandkumar, Cris Cecka

2016 IEEE 23rd International Conference on High Performance Computing (HiPC)

Tensor Contractions with Extended BLAS Kernels on CPU and GPU

Yang Shi, U. N. Niranjan, Animashree Anandkumar, Cris Cecka

2016 IEEE 23rd International Conference on High Performance Computing (HiPC)

Approxilyzer: Towards A Systematic Framework for Instruction-Level Approximate Computing and its Application to Hardware Resiliency

Radha Venkatagiri, Abdulrahman Mahmoud, Siva Hari, Sarita Adve

International Symposium on Microarchitecture (MICRO)

vDNN: Virtualized Deep Neural Networks for Scalable, Memory-Efficient Neural Network Design.

Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, Steve Keckler

International Symposium on Microarchitecture (MICRO)

All-Inclusive ECC: Thorough End-to-End Protection for Reliable Computer Memory

Jungrae Kim, Michael B. Sullivan, Sangkug Lym, Mattan Erez

The International Symposium on Computer Architecture (ISCA)

S-Step and Communication-Avoiding Iterative Methods

Maxim Naumov

Technical Report NVR-2016-003

Towards High Performance Paged Memory for GPUs

Tianhao Zheng, David Nellans, Arslan Zulfiqar, Mark Stephenson, Steve Keckler

International Symposium on High Performance Computer Architecture (HPCA)

A Case for Toggle-Aware Compression for GPU Systems

Gennady Pekhimenko, Evgeny Bolotin, Nandita Vijaykumar, Onur Mutlu, Todd C. Mowry, Steve Keckler

International Symposium on High Performance Computer Architecture (HPCA)

Selective GPU Caches to Eliminate CPU-GPU HW Cache Coherence

Neha Agarwal, David Nellans, Eiman Ebrahimi, Thomas F. Wenisch, John Danskin, Steve Keckler

International Symposium on High Performance Computer Architecture (HPCA)

Parallel Spectral Graph Partitioning

Maxim Naumov, Timothy Moon

Technical Report NVR-2016-001

2015

Network Endpoint Congestion Control for Fine-Grained Communication

Ted Jiang, Larry Dennison, William Dally

SC15

The Light Field Stereoscope

Fu-Chung Huang, David Luebke, Gordon Wetzstein

ACM SIGGRAPH 2015 Emerging Technologies

Parallel Graph Coloring with Applications to the Incomplete-LU Factorization on the GPU

Maxim Naumov, Patrice Castonguay, Jonathan Cohen

Technical Report NVR-2015-001

In-Memory Graph Databases for Web-Scale Data

Vito Giovanni Castellana, Alessandro Morari, Jesse Weaver, Antonino Time, David Haglin, Oreste Villa, John Feo

IEEE Computer

2014

Scaling the Power Wall: A Path to Exascale

Oreste Villa, Daniel Johnson, Mike O'Connor, Evgeny Bolotin, David Nellans, Justin Luitjens, Nikolai Sakharnykh, Peng Wang, Paulius Micikevicius, Anthony Scudiero, Steve Keckler, William Dally

SC '14