Sparse Attention Still Has a Dense Problem
Using the indexer’s own history to cut the cost of deciding where to attend.
Projects
Granary-v2: Scalable Multilingual Speech Data Curation for Predictable and Controllable Model Outputs
Published:
We present Granary-v2, a multilingual speech dataset and reusable curation system that provides a data foundation for speech-language models that follow instructions.
NVIDIA Nemotron 3.5 Transcribe: Intelligent Transcription with a Speech LLM
Published:
We introduce Nemotron 3.5 Transcribe, an open speech LLM for intelligent transcription: a single model that transcribes 40 languages verbatim or in written form, with context, speaker attribution, and word timestamps, selected with a natural language instruction. It is the best open-weights model on the English and multilingual Open ASR Leaderboards.
RIFT: KV Cache Compression for Draft Models with Complementary Compressibility Training
Published:
We introduce RIFT, a family of KV cache compression strategies for draft models, which we complement with a compression-aware knowledge distillation procedure.
NVIDIA Nemotron 3.5 Bolt E4B 4GB
Published:
We are releasing NVIDIA Nemotron 3.5 Bolt E4B 4GB, an open small language model optimized for speed and intelligence per GB in conversational agents.
NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB
Published:
We are releasing NVIDIA Nemotron 3.5 Bolt 74B-A9B 24GB, a Mixture-of-Experts hybrid Mamba-Transformer model designed to run everyday agentic tasks locally on NVIDIA RTX/DGX Spark.
NVIDIA Nemotron 3.5 Super and Lightning
Published:
NVIDIA Nemotron 3.5 Super and Lightning are open, efficient Mixture-of-Experts Hybrid Mamba-Transformer models designed for reasoning, long-context understanding, multimodal intelligence, and agentic AI.
LiLiCorr: Lightweight Likelihood Correlation of Parallel Drafts for Speculative Decoding
Published:
LiLiCorr correlates the per-position marginals of a parallel block drafter in a single network pass, turning coherence into matrix operations that parallelize efficiently. It is the fastest system in almost all the settings we evaluate, across benchmarks, serving concurrencies and input lengths.
NVIDIA Nemotron 3 Ultra
Published:
We are releasing NVIDIA Nemotron 3 Ultra - our largest and most capable model yet. Nemotron 3 Ultra is a 55B active 550B total parameter Mixture-of-Experts hybrid Mamba-Transformer model that leverages Latent MoE, includes MTP Layers, and was pre-trained in NVFP4.
Accelerating RL Post-Training with Speculative Decoding in NeMo RL
Published:
We integrate speculative decoding into NeMo RL with a vLLM backend to accelerate rollout generation while preserving verifier-side training semantics. On 8B reasoning workloads, this yields up to 1.8x faster rollout generation and up to 1.4x faster RL steps, with projected gains of roughly 2.5x at 235B scale.
Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
Published:
We introduce Nemotron-Cascade 2, an open 30B MoE model with 3B activated parameters that delivers best-in-class reasoning and strong agentic capabilities. It is the second open-weight LLM, after DeepSeek-V3.2-Speciale-671B-A37B, to achieve Gold Medal-level 🏅 performance in 2025 International Mathematical Olympiad (IMO), the International Olympiad in Informatics (IOI), and the ICPC World Finals.
NVIDIA Nemotron 3 Super
Published:
We are releasing NVIDIA Nemotron 3 Super, a 12B active 120B total parameter Mixture-of-Experts hybrid Mamba-Transformer model. Nemotron 3 Super is the first model in the Nemotron 3 series that leverages Latent MoE, includes MTP Layers, and was pre-trained in NVFP4.
Enable NVFP4 Inference for Nemotron with Quantization-Aware Distillation
Published:
QAD Tech Report
Nemotron-3-Nano-30B-A3B-NVFP4 Model Card
Think Smart About Sparse Compute: LatentMoE for Higher Accuracy per FLOP and per Parameter
Published:
Read the Paper
Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models
Published:
We scale up cascaded reinforcement learning (Cascade RL) to develop general purpose reasoning models, Nemotron-Cascade, capable of operating in both instruct and deep thinking modes. Our 14B model can outperform its SFT teacher and achieves silver-medal performance in IOI 2025.