Technical Report

Introducing NVIDIA Nemotron 3.5 Bolt E4B 4GB

Today, we’re excited to release Nemotron 3.5 Bolt E4B 4GB — our latest open small language model optimized for speed and intelligence per GB in conversational agents.

Bolt E4B builds on Gemma 4 E4B with Nemotron post-training. It combines natural conversation with multi-turn agentic capabilities to power voice/device assistants, interactive characters and robotic/IoT interfaces running on NVIDIA GeForce RTX, Jetson, RTX Spark and DGX Spark.

Bolt E4B offers ~40% higher average quality against our prior model on key conversational agent benchmarks and outputs up to ~80% fewer tokens vs peers.

Designed to pack conversational agent intelligence into as little as ~4GB in its target workloads, Bolt E4B enables responsive local experiences while preserving headroom for applications and other models in the same pipeline.

Model Features

• Multi-turn tool use: call application APIs to carry out tasks across a conversation, with integration guidance for asynchronous tool calls
• Conversational mode: toggle to encourage concise, speech-ready responses for natural back-and-forth interactions
• Configurable reasoning: enable reasoning for complex requests or disable it for faster responses, with integration options to cap thinking to meet latency targets
• 64k context: accommodate conversation history, instructions, tool definitions, retrieved information and application state
• Multilingual capability: support for 35 languages, building on a base model pre-trained on +140 languages

Agents On Every Device

We’re always learning from how NVIDIA Jetson, GeForce RTX and DGX Spark customers — enterprises and enthusiasts alike — use our hardware to accelerate their workloads.

As language models become more capable, we’re seeing growing customer interest in using them for experiences like:

• Device & Voice Assistants: Fast local assistants for PC, mobile, app, home or OS control; customer service; and in-cabin experiences for vehicles
• Virtual & Interactive Characters: In-game NPCs, companions, and other interactive characters that blend dialogue with actions while maintaining a persona
• Embedded & Industrial Agents: Human-machine interfaces for robotics and heavy machinery; system automation and industrial IoT

Reachy Mini running on Jetson Orin Nano 8GB. Cosmos 3 Edge supplies real-time visual understanding of the world to Bolt E4B, which calls scripts from the robot’s animation libraries and powers its spoken responses.

Testing Bolt E4B on a blacksmith trader NPC. The agent verifies correct payment for an item, then autonomously calls tools to check crafting recipes, collect ingredients, craft the item, and transfer it to the user.

Many large models are capable of powering these agents — but constraints around memory, latency, hardware compatibility, cost, or privacy concerns prevent developers from deploying these in production.

And so, they turn to faster, cheaper, smaller local models that can unlock these experiences.

Evaluating Real-Time Interactive Agents

Choosing a model for these experiences is not as simple as picking the highest score on today’s LLM leaderboards.

A model that excels at agentic coding is often not the best fit for a fast home assistant that’s fun to interact with — these agents need a different mix of capabilities. Through conversations with customers, several common characteristics emerge:

• The model must often embody an agent that converses naturally with a human in the same physical or virtual space
• It must know whether, when and how to interleave dialogue with correct tool calls to take actions
• It must reliably honor explicit behavioral, persona, brand, or output constraints
• It must understand its environment by interpreting supplied app/world telemetry, retrieved database payloads or tool results
• And, it must ground all dialogue and actions in the relevant parts of all that context — especially amid distractors or as that context grows throughout a conversation

Current benchmarks only partially capture what makes these agents work well, and no individual benchmark covers every capability an application needs. Ultimately, benchmark gains must translate into better behavior in the target application.

Model Benchmarks & Performance

We selected complementary benchmarks that closely reflect customer workloads or that customers already use, and grouped them into five capabilities. The chart below compares Bolt E4B with models of different sizes on representative benchmark quality, time to first token and VRAM usage:

Average Q4_K_M score across MultiChallenge, RoleMRC, BFCL-v3, IHEval, SysBench and RelyToolBench. Measured on RTX 5090 at ~3K/50 ISL/OSL. Cloud reference results (score/TTFT): Gemini 3.8 Flash (77.1 / 2,449.5ms); Claude Sonnet 5.5 (76.2 / 3,002.2ms).

Conversation & Persona

Multi-turn Tool Use

Instruction & Control

Context Robustness

State & Action Grounding

All results BF16. Every benchmark uses the same reasoning mode across models. Reasoning mode varies by benchmark. Brevity Bench and When2Call-MT address gaps in existing evaluations — to be released at a later date.

Across all these benchmarks, Bolt E4B achieves roughly 40% higher average scores than our prior model, Nano 4B, and roughly 10% higher scores than other peers evaluated.

These quality gains also come with fewer output tokens — up to 80% fewer in reasoning enabled comparisons. Reducing unnecessary generation lowers decoding work and costs, and can shorten response times.

Good responses still need to be fast responses in their target workloads — typically single session (batch size 1) envelopes between 2.5K and 16K input tokens and fewer than 500 output tokens.

The table below reports peak memory usage, time to first token, and generation throughput across representative hardware and workloads.

RTX

Inference performance across hardware. All configurations measured using Q4_K_M at batch size 1 on llama.cpp (CUDA) with AIPerf. Cache/ISL/OSL denotes cached context, new input, and output lengths in tokens. Results may vary with hardware, inference software, and configuration.

Jetson

Inference performance across hardware. All configurations measured using Q4_K_M at batch size 1 on llama.cpp (CUDA) with AIPerf. Cache/ISL/OSL denotes cached context, new input, and output lengths in tokens. Results may vary with hardware, inference software, and configuration.

Training Methodologies & Open Recipes

Bolt E4B builds on Gemma4-E4B-it through Nemotron post-training methods across environments targeting instruction following, multi-turn tool-calling and conversational capabilities.

We used reinforcement learning with verifiable rewards (RLVR) and from human feedback (RLHF), powered by NeMo-RL and NeMo-Gym. The training datasets are listed below, and the full recipe is available here:

• Tool-Calling: Conversational Tool Use Pivot
• Tool-call schema: Function Calling Pivot
• Instruction Following: Instruction Following, SysBench, InverseIFEval, CFBench, MultiTurnChat, Calendar
• Instruction Hierarchy: System-User-Instruction-Hierarchy
• Structured Outputs: Structured Outputs
• Free-form formatting: Free-Form Formatting
• GenRM: Updated version of Nemotron-RLHF-GenRM-v1

To emphasize non-reasoning accuracy, we used every sample in the RLVR/RLHF training blend in both reasoning and non-reasoning modes. We profiled rewards for each dataset separately in both modes, and maintained a 2:1 ratio of non-reasoning to reasoning samples in each training batch.

As post-training progressed, both reasoning traces and final answers became substantially longer. To control verbosity, we increased group-relative length penalties for reasoning and answer length in the GenRM environment. In all other environments, we disabled overlong filtering.

After post-training, we quantized the selected BF16 checkpoint to Q4_K_M using llama.cpp with importance-matrix (imatrix) calibration. Calibration used approximately 1M tokens of RL rollouts from the same training step as the selected checkpoint.

This release also includes a DFlash parallel drafter. It proposes all draft tokens in one forward pass, which the target verifies in parallel while preserving the target model’s output distribution. DFlash delivers up to 21% higher throughput relative to Gemma-4-E4B’s original MTP drafter.

Check out our technical report for more details!

Deploy it, Configure it, and Customize it for Your Use Case

Get started with Bolt E4B today through Hugging Face, Ollama, LM Studio, Unsloth, Canonical, Jetson AI Lab and the NVIDIA In-Game Inferencing SDK.

Explore our model card and deployment snippets, training recipes, and prompting/app setup guidelines.

We’re excited to see what you build!

Checkpoints

• NVIDIA-Nemotron-3.5-Bolt-E4B-4GB - Post-trained model in GGUF
• NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash - Drafter model for the GGUF checkpoint
• NVIDIA-Nemotron-3.5-Bolt-E4B-4GB—BF16 - Post-trained model in BF16 HF
• NVIDIA-Nemotron-3.5-Bolt-E4B-4GB-DFlash-NVFP4 - Drafter model for the HF checkpoint