Qwen3.8-Flash-Next Requirements: VRAM 75-354GB (2026)

To run Qwen3.8-Flash-Next locally, the memory footprint runs from about 75GB at 1-bit quantization to about 111GB at 4-bit, up to roughly 354GB at full BF16 precision. Alibaba's Qwen team released this open-weight MoE on August 26, 2026 — 125B total parameters with about 6B active per token, positioned as a preview of the Qwen4 architecture — with a native 262,144-token context extendable to 1,048,576 tokens via YaRN, plus multimodal input. The most striking finding: using MoE expert offloading (as implemented in llama.cpp-family tooling), the model has actually been run on a single 24GB-class consumer GPU (RTX 4090). At the same time, Alibaba's official vLLM/SGLang production serving recipe (FP8) budgets roughly 250GB of aggregate GPU memory and is not officially supported on a single H100 — or even a single H200 — making this a model with an unusually wide gap between quantized local use and the official production configuration.

Qwen3.8-Flash-Next centers on a sparse MoE with 125B total parameters and roughly 6B active per token, using a 48-layer hybrid of Gated DeltaNet and Qwen Sparse Attention (512 experts, with 10 routed plus 1 shared active at any time). It also carries a roughly 51B-parameter n-gram embedding table and a 4B-parameter MTP (Multi-Token Prediction) head; including these, the full checkpoint totals roughly 180B parameters, or about 354GB in BF16 — consistent with the actual Hugging Face repository size. Alibaba frames this release as an "experimental preview of the next-generation Qwen4 architecture," a different, efficiency-focused line from the 2.4T-class flagship Qwen3.8-Max. It is natively multimodal (image input), and licensed not under Apache-2.0 but the Qwen Community License 1.0 (free commercial use, but products exceeding 100M monthly active users or $20M monthly revenue must prominently display the model name, and MaaS/AI-work-assistant businesses need a separate license) — a different arrangement from the Apache-2.0 Qwen3.8-27B.

Because MoE (Mixture of Experts) models route each token through only a subset of experts, the "~6B active" figure gets emphasized as if it were the memory footprint. But in a setup that keeps the whole model resident on GPU, which experts get chosen changes token by token — so effectively every expert (the full 125B-equivalent parameter count, plus the n-gram table and other components) has to sit in memory, which is the basis for the ~354GB BF16 and ~111GB 4-bit figures. There is an exception, though: llama.cpp and similar implementations support "MoE expert offloading," which keeps infrequently used expert weights in system RAM and transfers only what's needed for each token to the GPU, dramatically cutting VRAM usage. On X (formerly Twitter), one user reported running Qwen3.8-Flash-Next on a single RTX 4090 (24GB VRAM) with a 250,000-token context, at 21 t/s decode and 364 t/s prefill (without MTP, dflash, or KV cache quantization). The key to understanding this model is thinking in terms of total effective memory — VRAM plus system RAM — rather than VRAM alone.

Apple Silicon's Unified Memory architecture, which draws no distinction between VRAM and system RAM, is a good fit for this model because it lets a quantized checkpoint load without needing to think about MoE offloading. Unsloth's official documentation recommends "at least 96GB of RAM/unified memory for practical use, even at the smallest 1-bit quantization (~74.5GB)." As a rule of thumb, an M3/M4 Max MacBook Pro (128GB ceiling) is workable for 1-3 bit quantization, while a Mac Studio in the 128GB-192GB range (M3/M4 Ultra class) is the realistic choice for running 4-bit (~111GB) reliably. MLX-format distributions are also referenced, and Metal-enabled llama.cpp builds work as well. Keep in mind that the actual quantized file size — which includes the n-gram table and MTP head — runs larger than a naive "4-bit means 63GB" calculation would suggest.

Beyond the native 262,144 tokens, YaRN-based RoPE scaling extends context up to 1,048,576 tokens. Using this extension on self-hosted deployments (vLLM/SGLang/TokenSpeed) requires explicitly editing rope_parameters in config.json or passing the relevant YaRN flags — it's not on by default (the hosted QwenCloud API defaults to a 1M context). As context grows, KV cache memory scales roughly linearly, so memory needs at full 262K versus full 1M usage can differ by tens of gigabytes. If you plan to run long context routinely, budget beyond the quantization table above. Alibaba reports up to 8.6x faster 1M-token prefill throughput versus the prior-generation Qwen3.7-Plus (vendor-reported figure).

On agentic tasks — SWE-bench Pro, CoWorkBench, JobBench — Qwen3.8-Flash-Next outperforms Claude Opus 4.6 Max, while on Humanity's Last Exam, a broad knowledge/reasoning benchmark, Opus scores higher. All figures are vendor-reported; real-world differences will depend on the evaluation set, prompting, and tool setup.

- Quantized local (llama.cpp / Unsloth Desktop): pull a quantized file such as UD-Q4_K_XL from unsloth/Qwen3.8-Flash-Next-GGUF, and run it with a Metal-enabled llama.cpp build or the Unsloth Desktop app; tune MoE offload flags to balance VRAM against RAM
- Apple Silicon / MLX: use an MLX-format build to load the quantized model across Unified Memory in one pool (96GB+ Mac recommended)
- Production serving (vLLM / SGLang): run Qwen/Qwen3.8-Flash-Next-FP8 on vLLM or SGLang. The official recipe assumes multiple GPUs (TP2+), so plan your configuration knowing a single GPU is unsupported
- API (fastest overall): obtain a QwenCloud API key and call Qwen3.8-Flash (the production API model) via the endpoint, with a 1M context enabled by default

- Load fails from insufficient VRAM: drop the quantization level (4-bit to 3-bit to 2-bit), or enable MoE offloading to push some expert weights to system RAM
- Extremely slow inference: too much is being offloaded to CPU; keep more layers/experts on GPU where possible to reduce GPU-CPU transfer frequency
- Out-of-memory at long context: KV cache is the bottleneck; stick to the native 262K context instead of 1M, or enable KV cache quantization where the engine supports it
- Errors trying to launch the official vLLM/SGLang recipe on a single GPU: the official recipe assumes multiple GPUs; for single-GPU use, switch to the GGUF quantized + llama.cpp-family path instead
- Mac shows heavy swapping and runs out of memory: lower the quantization level, or close resident apps to free up Unified Memory