Research Areas

Natural Language Processing

Associated Publications

2026

Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding
Yonggan Fu, Lexington Whalen, Abhinav Garg, Chengyue Wu, Maksim Khadkevich, Nicolai Oswald, Enze Xie, Daniel Egert, Sharath Turuvekere Sreenivas,, Shizhe Diao, Chenhan Yu, Ye Yu, Weijia Chen, Sajad Norouzi, Jingyu Liu, Shiyi Lan, Ligeng Zhu, Jin Wang, Jindong Jiang, Morteza Mardani, Mehran Maghoumi, Song Han, Ante Jukić, Nima Tajbakhsh, Jan Kautz, Pavlo Molchanov
TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models
Tong Guan, Huck Yang, Sabato Marco Siniscalchi, Qingsong Wen, Ming Jin, Shirui Pan
ICLR
RLP: Reinforcement as a Pretraining Objective
Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
International Conference on Learning Representations (ICLR) 2026
iGRPO: Self-Feedback-Driven LLM Reasoning
Ali Hatamizadeh, Shrimai Prabhumoye, Igor Gitman, Ximing Lu, Seungju Han, Wei Ping, Yejin Choi, Jan Kautz

2025

Alpamayo 1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
Marco Pavone, Many other contributors found on Page 33
Fugatto 1 - Foundational Generative Audio Transformer Opus 1
Rafael Valle, Rohan Badlani, Zhifeng Kong, Sang-gil Lee, Arushi Goel, Sungwon Kim, Joao Felipe Santos, Shuqi Dai, Siddharth Gururani, Aya AIJa'fari, Alex Liu, Kevin Shih, Wei Ping, Huck Yang, Bryan Catanzaro
ICLR 2025
Gated Delta Networks: Improving Mamba2 with Delta Rule
Songlin Yang, Jan Kautz, Ali Hatamizadeh
International Conference on Learning Representations (ICLR) 2025
Hymba: A Hybrid-head Architecture for Small Language Models
Xin Dong, Yonggan Fu*, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van keirsbilck, Min-Hung Chen, Yoshi Nishi, Yingyan Celine Lin, Jan Kautz, Pavlo Molchanov
Hymba - ICLR 2025
ICLR spotlight paper
Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Marcin Chochowski, Yashaswi Karnati, Raviraj Joshi, Ameya Sunil Mahabaleshwarkar, Zijia Chen, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov
NeurIPS 2025
Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, Huck Yang, EngSiong Chng
ICLR 2025
Towards Neural Scaling Laws for Time Series Foundation Models
Qingren Yao, Huck Yang, Renhe Jiang, Ming Jin, Shirui Pan
ICLR 2025
Semantic Prompt Learning for Weakly-Supervised Semantic Segmentation
Ci-Siang Lin, Chien-Yi Wang, Frank Wang, Min-Hung Chen
Winter Conference on Applications of Computer Vision (WACV)
Spatio-Temporal Context Prompting for Zero-Shot Action Detection
Wei-Jhe Huang, Min-Hung Chen, Shang-Hong Lai
Winter Conference on Applications of Computer Vision (WACV)

2024

Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
Huck Yang, Taejin Park, Yuan Gong, Yuanchao Li, Zhehuai Chen, yen-ting Lin, Chen Chen, Yuchen Hu, Kunal Dhawan, Piotr Zelasko, Chao Zhang, Yun-Nung Chen, Yu Tsao, Jagadeesh Balam, Boris Ginsburg, Shinji Watanabe, Andreas Stolcke
SLT 2024
Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models
Yuchen Hu, Chen Chen, Huck Yang, Chengwei Qin, Pin-Yu Chen, Eng Siong Chng, Chao Zhang
NeurIPS
From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichment
Yusuke Hirota, Ryo Hachiuma, Huck Yang, Yuta Nakashima
EMNLP
Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
Siyin Wang, Huck Yang, Ji Wu, Chao Zhang
EMNLP
FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Model
Yichen Lu, Jiaqi Song, Huck Yang, Shinji Watanabe
EMNLP
HAMSTER: Hierarchical Action Models for Open-World Robot Manipulation
Yi Li, Yuquan Deng, Jesse Zhang, Joel Jang, Marius Memmel, Caelan Garrett, Fabio Ramos, Dieter Fox, Anqi Li, Abhishek Gupta, Ankit Goyal
International Conference on Learning Representations (ICLR)
Guiding Long-Horizon Task and Motion Planning with Vision Language Models
Zhutian Yang, Caelan Garrett, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling
IEEE International Conference on Robotics & Automation (ICRA)
Open-World Task and Motion Planning via Vision-Language Model Inferred Constraints
Nishanth Kumar, William Shen, Fabio Ramos, Dieter Fox, Tomás Lozano-Pérez, Leslie Pack Kaelbling, Caelan Garrett
CoRL 2024 Workshop on Language and Robot Learning Language as an Interface
GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators
Yuchen Hu, Chen Chen, Huck Yang, Ruizhe Li, Zhehuai Chen, Eng Siong Chng
ACL 2024
DoRA: Weight-Decomposed Low-Rank Adaptation
Shih-Yang Liu, Chien-Yi Wang, Hongxu Danny Yin, Pavlo Molchanov, Frank Wang, Kwang-Ting Cheng, Min-Hung Chen
International Conference on Machine Learning (ICML) 2024
An Empirical Study of Mamba-based Language Models
Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, Bryan Catanzaro
https://arxiv.org/pdf/2406.07887
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
YuChen Hu, Chen Chen, Huck Yang, Ruizhe Li, Chao Zhang, Pin-Yu Chen, EnSiong Chng
ICLR 2024
It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognition
Chen Chen, Ruizhe Li, Yuchen Hu, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng, Huck Yang
ICLR 2024
A Chat about Boring Problems: Studying GPT-Based Text Normalization
Yang Zhang, Travis M. Bartley, Mariana Graterol-Fuenmayor, Vitaly Lavrukhin, Evelina Bakhturina, Boris Ginsburg
ICASSP

2023

HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models
Chen Chen, YuChen Hu, Huck Yang, Sabato Marco Siniscalchi, Pin-Yu Chen, Ensiong Chng
NeurIPS 2023
Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition
Srijith Radhakrishnan, Huck Yang, Sumeer Khan, Rohit Kumar, Narsis Kiani, David Gomez-Cabrero, Jesper Tegnér
EMNLP
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails
Traian Rebedea, Razvan Dinu, Makesh Sreedhar, Christopher Parisien, Jonathan Cohen
SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF
Yi Dong, Zhilin Wang, Makesh Narsimhan Sreedhar, Xianchao Wu, Oleksii Kuchaiev

2022

Evaluating Parameter Efficient Learning for Generation
Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan J. Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro
Thutmose Tagger: Single-pass neural model for Inverse Text Normalization
Alexandra Antonova, Evelina Bakhturina, Boris Ginsburg
Shallow Fusion of Weighted Finite-State Transducer and Language Model for Text Normalization
Evelina Bakhturina, Yang Zhang, Boris Ginsburg

2021

Text Mining Drug/Chemical-Protein Interactions using an Ensemble of BERT and T5 Based Models
Virginia Adams, Hoo-Chang Shin, Carol Anderson, Bo Liu, Anas Abidin
A Unified Transformer-based Framework for Duplex Text Normalization
Tuan Manh Lai, Yang Zhang, Evelina Bakhturina , Boris Ginsburg, Heng Ji
SGD-QA: Fast Schema-Guided Dialogue State Tracking for Unseen Services
Yang Zhang, Vahid Noroozi, Evelina Bakhturina, Boris Ginsburg
NeMo Inverse Text Normalization: From Development To Production
Yang Zhang, Evelina Bakhturina, Kyle Gorman, Boris Ginsburg

2020

BioMegatron: Larger Biomedical Domain Language Model
Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
ACL Anthology
A Fast and Robust BERT-based Dialogue State Tracker for Schema-Guided Dialogue Dataset
Vahid Noroozi, Yang Zhang, Evelina Bakhturina, Tomasz Kornuta

Researchers

Ali Hatamizadeh
Chia-Wen Kuo
Ekta Prashnani
Fengyuan Hu
Jaesung Choe
Sameer Dharur
Shizhe Diao
Sung-Feng Huang
Ximing Lu
Yejin Choi
Yoad Tewel
Yonggan Fu
Yuqi Xie
Yusuke Hirota