Nemotron 3 Requirements: VRAM & GPUs for Nano/Super/Ultra

NVIDIA's open-weight Nemotron 3 family comes in three sizes — Nano (31.6B total / ~3.2B active), Super (120B total / 12B active) and Ultra (550B total / 55B active). At 4-bit class quantization (NVFP4), inference memory lands at roughly 18GB, 69GB and 317GB respectively. At full BF16 precision those figures jump to about 73GB, 276GB and 1.27TB. All three are Mixture-of-Experts models, but a small active parameter count does not reduce memory: every expert's weights must stay resident during inference, so memory is governed by total parameters.

The practical conclusion first: Nano runs on a single consumer GPU, while Super and Ultra assume multi-GPU data-centre hardware. The details follow.

These figures add roughly 15% inference overhead (KV cache, activations) on top of raw weight size. Nemotron 3 supports up to 1M tokens of context, but actually using that much pushes KV cache well beyond this estimate — which is why the official model cards default to 262,144 tokens (256k).

Beyond estimates, NVIDIA and vLLM publish concrete serving configurations. These are the reliable baseline for production planning.

Running Ultra on 8x H100 (640GB combined) requires NVFP4; BF16 needs sixteen cards. On Blackwell B200 hardware, NVFP4 fits in four cards, more than halving the GPU count. NVFP4 is Blackwell's native format and Nemotron 3 was pretrained in it, so the quality loss is described as smaller than post-hoc 4-bit quantization.

Nemotron 3 is NVIDIA's open-weight LLM family, distinguished from most open-weight releases by publishing training data and recipes alongside the weights. It ships under the NVIDIA Nemotron Open Model License, which permits commercial use.

Architecturally it is a hybrid of Mamba-2 and Transformer layers combined with Latent MoE. Mamba-2 layers handle sequence processing in linear time, with attention layers interleaved where precise reasoning is needed, which keeps compute growth in check even at 1M-token context. Multi-Token Prediction layers are also present, with NVIDIA reporting up to 3x wall-clock speedups for structured generation.

Nano has 31.6B total and about 3.2B active parameters, released in December 2025. At 4-bit it fits in roughly 18GB, so a single RTX 4090 (24GB) or RTX 5090 (32GB) suffices. FP8 at around 36GB calls for an RTX 6000 Ada class card (48GB) or two 24GB cards. BF16 at around 73GB realistically means Apple Silicon with 96GB or 128GB unified memory, or one H100.

On performance, NVIDIA reports roughly 3.3x the throughput of Qwen3-30B on a single H200. Because the active parameter count sits in the 3B range, generation speed tends to beat dense models of comparable footprint once memory is secured.

Super is 120B total / 12B active, released on 11 March 2026 and positioned for high-volume workloads such as collaborative agents and IT ticket automation. The stated BF16 minimum is 8x H100-80GB, but the NVFP4 variant is documented as running on a single B200 or on DGX Spark — and that ability to fit in one box is Super's main practical appeal.

NVIDIA claims 5x the throughput of the previous-generation Nemotron Super and 4x better memory and compute efficiency versus FP8 on H100. vLLM support is official from version 0.18.1, with documented settings including --tensor-parallel-size 8, --kv-cache-dtype fp8 and --max-model-len 262144.

Ultra is 550B total / 55B active, published to Hugging Face on 4 June 2026. It scores 47.7 on the Artificial Analysis Intelligence Index, the strongest US-built open-weight model on that measure. vLLM shipped day-0 support via the vllm/vllm-openai:v0.22.0 container, with architecture-specific flags such as --mamba-backend triton and --reasoning-parser nemotron_v3.

The open-weight frontier is otherwise dominated by Chinese releases — GLM-5.2 (753B), Kimi K3 (2.8T), DeepSeek V4 and Qwen3.6 — and Ultra is best read as the US answer to them. For comparison at similar scale, see our GLM-5.2 requirements reference and Inkling requirements guide.

Three options are practical for Nemotron 3: BF16, FP8 and NVFP4. NVFP4 is a 4-bit floating point format with hardware support on Blackwell (B200/GB200/GB300/B300/RTX PRO Blackwell); it will run on Hopper (H100/H200) but without the intended speed. Ampere (A100) lacks FP8 hardware support, so it effectively means BF16 operation and double the memory.

Individual or small-scale testing (under ~USD 3,500): Nano at 4-bit on a single RTX 4090/5090. 24GB is enough unless you need 1M context
Workstation (~USD 7,000–20,000): Nano at BF16 or Super at NVFP4 on a Mac Studio with 128GB+ or DGX Spark. The ceiling for single-box deployment
Single server (from ~USD 70,000): Super at NVFP4 on one B200, or Super at BF16 on 8x H100
Cluster (from ~USD 200,000): Ultra at NVFP4 on 4x B200 or 8x H100; BF16 operation needs 8x B200 or 16x H100

To check whether a specific model fits your own hardware, our VRAM calculator derives required memory from model size, quantization and context length. For comparison against lighter models, see our Gemma 4 hardware requirements reference.

The first Nemotron-specific pitfall is that the Mamba-2 hybrid architecture requires dedicated vLLM settings. The official commands include --mamba-backend triton and --mamba-ssm-cache-dtype float32; omit them and serving either fails to start or underperforms. SSM state cache also consumes memory separately from the KV cache, so leave headroom beyond the table estimates above.

When loading fails on memory, work through these in order of least impact: (1) reduce context length from 1M to 256k or lower, (2) compress the KV cache with --kv-cache-dtype fp8, (3) lower concurrent sequences via --max-num-seqs, and (4) drop one precision tier. Even NVIDIA's own Ultra sample uses conservative values — 16 sequences and 32,768 batched tokens.