Inkling Requirements: VRAM, GPU & RAM by Quant (2026)

Running Inkling requires roughly 1.9TB of combined VRAM/RAM at full BF16 precision, or as little as 280–295GB at the lightest 1-bit quantization. Even at 4-bit quantization, around 600GB is the baseline. None of these configurations fit on a standard consumer PC or laptop. Inkling is a 975B total-parameter, 41B-active open-weight Mixture-of-Experts (MoE) model that assumes a multi-GPU server, a 512GB-class high-RAM server, or a high-unified-memory machine such as a Mac Studio Ultra.

Thinking Machines also ships an official NVIDIA Blackwell-optimized NVFP4 quantized checkpoint, which needs roughly 600GB minimum VRAM, while BF16 requires roughly 2TB of aggregate VRAM.

Inkling is Thinking Machines Lab's first open-weight model, announced on 2026-07-15. It's a sparse Mixture-of-Experts model built on a 66-layer decoder-only transformer, with 975B total parameters and roughly 41B active parameters per token (6 experts activated per token). It's multimodal, accepting text, image, and audio input, with a context window of up to 1,048,576 tokens (1M). It was trained on 45 trillion tokens of text, image, audio, and video, and includes a "controllable thinking effort" feature. It's released under the Apache 2.0 license, allowing free fine-tuning and commercial use with no royalties, and is downloadable via Hugging Face and Thinking Machines' own Tinker platform. For a contrast with a laptop-friendly model, see the Gemma 4 requirements reference.

Quantization reduces memory usage by lowering the number of bits used to represent a model's weights. For Inkling, compressing from full BF16 (16-bit) precision down to 1-bit cuts required memory from 1.9TB to 280–295GB, but accuracy retained also drops from 100% to 74.2–77.4%. The general tradeoff between bit-depth, accuracy, and memory is covered in more depth in our quantization and hardware requirements guide. 4-bit quantization, at 94.4% accuracy retained, is often the practical sweet spot between memory savings and quality.

A single consumer-class GPU in the 24–32GB range cannot hold Inkling. Even the lightest 1-bit build needs about 280GB, so a real deployment requires one of the following.

Metal is supported by default on Apple Silicon, with no GPU-specific flags required. However, the 1-bit quantized model still needs roughly 290GB or more of combined RAM+VRAM, which in practice means a Mac Studio Ultra-class machine with high unified memory. When VRAM is limited, partial GPU offload is possible via the --n-gpu-layers option.

Inkling can be deployed on major inference engines including SGLang, vLLM, TokenSpeed, and llama.cpp. Dynamic GGUF quantizations (1-bit and 2-bit) are available for llama.cpp, and a native NVFP4 quantized checkpoint is provided for NVIDIA Blackwell hardware.

Among open-weight MoE models, Inkling (975B total parameters) is still smaller than the even larger Kimi K3 (2.8T total parameters), but both are server-class models that don't run in typical consumer environments. By contrast, Gemma 4 is a lightweight model that runs on laptops — Inkling instead trades that portability for scale: multimodal input, a 1M-token context window, and 975B parameters, aimed squarely at server and high-end workstation deployments.