People

Steve Dai
Main Field of Interest
Circuits and VLSI Design

Publications

2023

Efficient Transformer Inference with Statically Structured Sparse Attention
Steve Dai, Hasan Genc, Rangharajan Venkatesan, Brucek Khailany
2023 60th ACM/IEEE Design Automation Conference (DAC)
A 95.6-TOPS/W Deep Learning Inference Accelerator With Per-Vector Scaled 4-bit Quantization in 5 nm
Ben Keller, Rangharajan Venkatesan, Steve Dai, Stephen Tell, Brian Zimmer, Charbel Sakr, William Dally, Tom Gray, Brucek Khailany
Journal of Solid-State Circuits

2022

LNS-Madam: Low-Precision Training in Logarithmic Number System Using Multiplicative Weight Update
Jiawei Zhao, Steve Dai, Rangharajan Venkatesan, Brian Zimmer, Mustafa Ali, Ming-Yu Liu, Brucek Khailany, William Dally, Anima Anandkumar
IEEE Transactions on Computers (Volume: 71, Issue: 12, 01 December 2022)
Optimal Clipping and Magnitude-aware Differentiation for Improved Quantization-aware Training
Charbel Sakr, Steve Dai, Rangharajan Venkatesan, Brian Zimmer, Brucek Khailany, William Dally
2022 International Conference on Machine Learning (ICML)
A 17–95.6 TOPS/W Deep Learning Inference Accelerator with Per-Vector Scaled 4-bit Quantization for Transformers in 5nm
Ben Keller, Rangharajan Venkatesan, Steve Dai, Stephen Tell, Brian Zimmer, William Dally, Tom Gray, Brucek Khailany
2022 Symposium on VLSI Technology & Circuits Digest of Technical Papers

2021

Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers
Jacob R. Stevens, Rangharajan Venkatesan, Steve Dai, Brucek Khailany, Anand Raghunathan
Design Automation Conference (DAC) 2021
VS-QUANT: Per-Vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
Steve Dai, Rangharajan Venkatesan, Haoxing (Mark) Ren, Brian Zimmer, William Dally, Brucek Khailany
MLSys 2021
Verifying High-Level Latency-Insensitive Designs with Formal Model Checking
Steve Dai, Alicia Klinefelter, Haoxing (Mark) Ren, Rangharajan Venkatesan, Ben Keller, Nathaniel Pinckney, Brucek Khailany
arXiv

2020

Accelerating Chip Design with Machine Learning
Brucek Khailany, Haoxing (Mark) Ren, Steve Dai, Saad Godil, Ben Keller, Robert Kirby, Alicia Klinefelter, Rangharajan Venkatesan, Yanqing Zhang, Bryan Catanzaro, William Dally
IEEE Micro

2019

MAGNet: A Modular Accelerator Generator for Neural Networks
Rangharajan Venkatesan, Sophia Shao, Miaorong Wang, Jason Clemons, Steve Dai, Matt Fojtik, Ben Keller, Alicia Klinefelter, Nathaniel Pinckney, Priyanka Raina, Yanqing Zhang, Brian Zimmer, William Dally, Joel Emer, Steve Keckler, Brucek Khailany
International Conference On Computer Aided Design (ICCAD)