Wenhao Ding

I am a research scientist in the autonomous vehicle research group at Nvidia. I'm interested in driving scenario generation, reinforcement learning, and causal discovery. I got my Bachelor's degree from Tsinghua University in 2018 and my Ph.D. from Carnegie Mellon University in 2024. 

Check my website for more information: https://wenhao.pub


 

Christos Kozyrakis

Christos' research focuses on computer architecture and systems software. He is currently working on cloud computing technology, systems design for artificial intelligence, and artificial intelligence for systems design. Christos holds a BS degree from the University of Crete (Greece) and a PhD degree from the University of California at Berkeley (USA). He is a fellow of the ACM and the IEEE.

Edward Suh

G. Edward Suh is a Senior Director of Research, and leads a group in security and privacy research.

He is also an Adjunct Professor in the School of Electrical and Computer Engineering at Cornell University, where he served on the faculty from 2007 to 2023. Before joining NVIDIA, he was a Research Scientist in the Fundamental AI Research (FAIR) team at Meta. He earned a B.S. in Electrical Engineering from Seoul National University and an M.S. and a Ph.D. in Electrical Engineering and Computer Science from the Massachusetts Institute of Technology (MIT). 

Jiaojiao Fan

Jiaojiao receives her PhD in Machine Learning from the Georgia Institute of Technology. Her early research focused on scaling neural network-based optimal transport and MCMC sampling, with recent work emphasizing controllable generation in generative image and video models. Her contributions have been recognized through publications at leading conferences like ICML, AISTATS, and COLT.

DoRA: Weight-Decomposed Low-Rank Adaptation

In this ICML'24 Oral paper, we first introduce a novel weight decomposition analysis to investigate the inherent differences between FT and LoRA. Aiming to resemble the learning capacity of FT from the findings, we propose Weight-Decomposed LowRank Adaptation (DoRA). DoRA decomposes the pre-trained weight into two components, magnitude and direction, for fine-tuning, specifically employing LoRA for directional updates to efficiently minimize the number of trainable parameters.