Publications

Our publications provide insight into some of our leading-edge research.

Filters

Filters

Publication Year

Research Areas

Events

2023

Efficient Transformer Inference with Statically Structured Sparse Attention
Steve Dai, Hasan Genc, Rangharajan Venkatesan, Brucek Khailany
A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm
Ben Keller, Rangharajan Venkatesan, Steve Dai, Stephen Tell, Brian Zimmer, Charbel Sakr, William Dally, Tom Gray, Brucek Khailany

2022

HEAT: Hardware-Efficient Automatic Tensor Decomposition for Transformer Compression
Jiaqi Gu, Ben Keller, Jean Kossaifi, Anima Anandkumar, Brucek Khailany, David Z. Pan
Spotlight Paper
LNS-Madam: Low-Precision Training in Logarithmic Number System Using Multiplicative Weight Update
Jiawei Zhao, Steve Dai, Rangharajan Venkatesan, Brian Zimmer, Mustafa Ali, Ming-Yu Liu, Brucek Khailany, William Dally, Anima Anandkumar
Demystifying Map Space Exploration for NPUs
Sheng-Chun Kao, Angshuman Parashar, Po-An Tsai, Tushar Krishna
Self Adaptive Reconfigurable Arrays (SARA): Learning Flexible GEMM Accelerator Configuration and Mapping-space using ML
Ananda Samajdar, Eric Qin, Michael Pellauer, Tushar Krishna
DiGamma: Domain-aware Genetic Algorithm for HW-Mapping Co-optimization for DNN Accelerators
Sheng-Chun Kao, Michael Pellauer, Angshuman Parashar, Tushar Krishna
Marvel: A Data-Centric Approach for Mapping Deep Learning Operators on Spatial Accelerators
Prasanth Chatarasi, Hyoukjun Kwon, Angshuman Parashar, Michael Pellauer, Tushar Krishna, Vivek Sarkar

2021

Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers
Jacob R. Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany, Anand Raghunathan
Optimizing Selective Protection for CNN Resilience
Abdulrahman Mahmoud, Siva Hari, Christopher W. Fletcher, Sarita V. Adve, Charbel Sakr, Naresh Shanbhag, Pavlo Molchanov, Michael B. Sullivan, Timothy Tsai, Steve Keckler
Union: A Unified HW-SW Co-Design Ecosystem in MLIR for Evaluating Tensor Operations on Spatial Accelerators
Geonhwa Jeong, Gokcen Kestor, Prasanth Chatarasi, Angshuman Parashar, Po-An Tsai, Sivasankaran Rajamanickam, Roberto Gioiosa, Tushar Krishna
Simba: scaling deep-learning inference with chiplet-based architecture
Yakun Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matt Fojtik, Ted Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Stephen Tell, Yanqing Zhang, William Dally, Joel Emer, Tom Gray, Brucek Khailany, Steve Keckler
ACM Research Highlight
GAMMA: Exploiting Gustavson’s Algorithm to Accelerate Sparse Matrix Multiplication
Guowei Zhang, Nithya Attaluri, Joel Emer, Daniel Sanchez
VS-QUANT: Per-Vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
Steve Dai, Rangharajan Venkatesan, Haoxing (Mark) Ren, Brian Zimmer, William Dally, Brucek Khailany
Learning Sparse Matrix Row Permutations for Efficient SpMM on GPU Architectures
Atefeh Mehrabi, Donghyuk Lee, Niladrish Chatterjee, Danial J. Sorin, Benjamin C. Lee, Mike O'Connor
Making Convolutions Resilient via Algorithm-Based Error Detection Techniques
Siva Hari, Michael B. Sullivan, Timothy Tsai, Steve Keckler
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads
Hyoukjun Kwon, Liangzhen Lai, Michael Pellauer, Tushar Krishna, Yu-Hsin Chen, Vikas Chandra
Flexion: A Quantitative Metric for Flexibility in DNN Accelerators
Hyoukjun Kwon, Michael Pellauer, Angshuman Parashar, Tushar Krishna

2020

HarDNN: Fine-Grained Vulnerability Evaluation and Protection for Convolutional Neural Networks
Abdulrahman Mahmoud, Siva Hari, Christopher W. Fletcher, Sarita V. Adve, Charbel Sakr, Naresh Shanbhag, Pavlo Molchanov, Michael B. Sullivan, Timothy Tsai, Steve Keckler
Planaria: Dynamic Architecture Fission for Spatial Multi-Tenant Acceleration of Deep Neural Networks
Soroush Ghodrati, Byung Hoon Ahn, Joon Kyung Kim, Sean Kinzer, Brahmendra Reddy Yatham, Navateja Alla, Hardik Sharma, Mohammad Alian, Eiman Ebrahimi, Nam Sung Kim, Cliff Young, Hadi Esmaeilzadeh
PyTorchFI: A Runtime Perturbation Tool for DNNs
Abdulrahman Mahmoud, Neeraj Aggarwal, Alex Nobbe, Jose Rodrigo Sanchez Vicarte, Sarita V. Adve, Christopher W. Fletcher, Iuri Frosio, Siva Hari
Making Convolutions Resilient via Algorithm-Based Error Detection Techniques
Siva Hari, Michael B. Sullivan, Timothy Tsai, Steve Keckler
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs
Esha Chouske, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Steve Keckler
An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives
Benjamin Klenk, Ted Jiang, Greg Thorson, Larry Dennison
MAESTRO: A Data-Centric Approach to Understand Reuse, Performance, and Hardware Cost of DNN Mappings
Hyoukjun Kwon, Prasanth Chatarasi, Vivek Sarkar, Tushar Krishna, Michael Pellauer, Angshuman Parashar
HarDNN: Feature Map Vulnerability Evaluation in CNNs
Abdulrahman Mahmoud, Siva Hari, Christopher W. Fletcher, Sarita V. Adve, Charbel Sakr, Naresh Shanbhag, Pavlo Molchanov, Michael B. Sullivan, Timothy Tsai, Steve Keckler

2019

MAGNet: A Modular Accelerator Generator for Neural Networks
Rangharajan Venkatesan, Sophia Shao, Miaorong Wang, Jason Clemons, Steve Dai, Matt Fojtik, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Yanqing Zhang, Brian Zimmer, William Dally, Joel Emer, Steve Keckler, Brucek Khailany
Understanding Reuse, Performance, and Hardware Cost of DNN Dataflows: A Data-Centric Approach.
Hyoukjun Kwon, Prasanth Chatarasi, Michael Pellauer, Angshuman Parashar, Vivek Sarkar, Tushar Krishna
IEEE Micro Top Picks in Computer Architecture
Simba: Scaling Deep-Learning Inference with Multi-Chip-Module-Based Architecture
Sophia Shao, Jason Clemons, Rangharajan Venkatesan, Brian Zimmer, Matt Fojtik, Ted Jiang, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Stephen Tell, Yanqing Zhang, William Dally, Joel Emer, Tom Gray, Brucek Khailany, Steve Keckler
Best Paper award
IEEE Micro Top Picks in Computer Architecture (Honorable Mention)
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs
Esha Choukse, Michael B. Sullivan, Mike O'Connor, Mattan Erez, Jeff Pool, David Nellans, Stephen W. Keckler
DeLTA: GPU Performance Model for Deep Learning Applications with In-depth Memory System Traffic Analysis
Sankug Lym, Donghyuk Lee, Niladrish Chatterjee, Mike O'Connor, Mattan Erez
Timeloop: A Systematic Approach to DNN Accelerator Evaluation
Angshuman Parashar, Priyanka Raina, Yakun Sophia Shao, Yu-Hsin Chen, Victor A. Ying, Anurag Mukkara, Rangharajan Venkatesan, Brucek Khailany, Steve Keckler, Joel Emer