Nemotron 3.5 Transcribe Explained

Nemotron 3.5 Transcribe is a prompt-controllable multilingual speech recognition (ASR) model that NVIDIA released on October 28, 2026. It is a 34B-parameter mixture-of-experts model with about 4.3B active, and its model card lists Japanese as supported for transcription, translation to and from English, and multi-speaker ASR (rated production-ready). The license is OpenMDW-1.1, covering commercial and non-commercial use. The recommended setup is one H100 80GB, and quantized builds (FP8 and NVFP4) extend support to GPUs such as the RTX 5090. It targets engineers who want accurate, self-hosted transcription of recordings up to one hour.

Nemotron 3.5 Transcribe (checkpoint nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) pairs a speech encoder with a hybrid Mamba-2 and Transformer mixture-of-experts language model. The Hugging Face blog says "30B, about 4B active," while the model card says "34B, about 4.3B active, including the audio encoder." The blog does not explain the gap, so this article mainly uses the model card numbers.

Where classic ASR simply turns audio into text, this model switches task and output style through natural-language instructions. NVIDIA also released Granary v2, a public dataset derived from the data family used in training.

- Switch between transcription, translation, and audio question answering (trained in English) with a prompt
- Choose a style such as verbatim (fillers and repetitions kept) or clean text
- Specify written forms and pass term lists for domain vocabulary
- Word-level timestamps
- Speaker-labeled transcripts (up to eight speakers)
- Recordings up to one hour in one pass (the blog says it is built for recorded audio, not live streaming)

According to the model card, input is 16 kHz mono in .wav, .flac, .mp3, or .ogg, with a 65,536-token context. One example prompt on the card reads "Audio transcription. Language: as spoken. Wording: verbatim, including fillers and repetitions." The full version, which also covers punctuation, capitalization, and transcript-only output, is on the card.

The blog describes 40 languages for transcription, 31 for translation to or from English, and 13 for speaker attribution. It does not print the language list and points to the model card's task-by-language matrix instead. In that matrix, Japanese is marked for ASR, translation with English, multi-speaker ASR, and code-switching, at the production-ready level. The multi-speaker languages are Arabic, Mandarin Chinese, English, French, German, Hindi, Italian, Japanese, Korean, Portuguese, Russian, Spanish, and Thai.

However, I could not find a Japanese-only WER or CER (character error rate) in the blog text or the model card portions I read. Japanese accuracy cannot be judged from published figures, so test it on your own recordings. For contrast with a small model that lacks Japanese, see the Whistle explainer.

The OpenMDW-1.1 license comes from the model card; the blog does not mention a license. Check the license text at the source before any commercial use. For Granary v2, the blog tells users to obtain source audio under each corpus's original license and follow the release's published terms.

No VRAM figure is stated officially. The model card gives a recommended setup (one H100 80GB) and a compatibility table. The table lists A100 80GB, H100, H200, B200, GB200, GB300, RTX PRO 6000 Blackwell, DGX Spark, and Jetson Thor, plus RTX 5090 (NVFP4 only) and L40S (FP8 only).

The figures below are my own estimates from the parameter count, not measurements, and they exclude KV cache and activations. Weights alone come to about 68GB in BF16 (34B x 2 bytes), about 34GB in FP8, and a little over 17GB in NVFP4. With MoE, all experts must sit in memory even though few are active. The same reasoning applies in the Nemotron 3.5 Bolt requirements article.

The model card shows several loaders. With NeMo, use nemo_asr.models.ASRModel.from_pretrained("nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B"). With Transformers, use AutoModelForMultimodalLM.from_pretrained(..., device_map="auto"). For a server, vLLM is the shortest path.

- 1. Install vLLM 0.32.0 and the NeMo speech plugin that the model card points to
- 2. Set the environment variable VLLM_PLUGINS=nemo_speechlm
- 3. Start the server with vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B (full command and arguments are on the model card)
- 4. Send audio and a prompt from an OpenAI-compatible client (the full Python client is also on the card)
- 5. Tune the prompt for verbatim or clean output, term lists, and timestamps

Hosted options exist too: the blog lists endpoints from Baseten, DeepInfra, and Simplismart. It also mentions running quantized builds locally through NeMo-Speech.cpp.

All numbers below are published by NVIDIA (model card or blog). They come from different measurement setups, so do not compare them across rows.

The blog also reports an Indian medical-speech case study. Out of the box the WER was 30.8%, a better prompt brought it to 23.8%, and fine-tuning (about 14 GPU-hours on one GB300) reached 11.8%. LibriSpeech test-clean worsened slightly from 1.54% to 1.69%, a reminder that specialization has a cost.

I found no direct comparison against OpenAI's Whisper large-v3 or turbo in the primary sources, so those cells are marked with a dash. The comparison targets below are the ones NVIDIA itself presents: Qwen3-ASR, Parakeet, and Canary-Qwen.

The main differences are size and controllability. It is far larger than 0.6B to 1.7B models and needs a GPU, but one model handles verbatim or clean output, vocabulary boosting, and speaker labels through prompts. For small, private, on-device dictation, a local tool such as TypeWhisper fits better. For turning transcripts into minutes, see the meeting minutes AI guide.

- Long recordings that need speaker labels: meetings, call centers, interviews
- Specialized domains (medical, legal) where term lists and fine-tuning can tighten accuracy
- Recordings mixing Japanese and other languages (code-switching is listed)
- Not suited to live captioning (the blog says recorded audio only)
- Too heavy for CPU-only PCs or phones