Inkling 2 Mini Requirements
26.4B MoE in ~20GB, Runs on a 32GB Mac or GPU (2026)
Inkling 2 Mini: 26.4B MoE (3.4B active), ~20GB checkpoint. 32GB+ memory/VRAM advised; GGUF from 18GB. Apache 2.0. Quant memory, Mac, GPU, how to run. Nov 2026
Inkling 2 Mini (thinkingmachines/Inkling-2-Mini) is an MoE model with 26.4B total and 3.4B active parameters per token, and its official checkpoint is about 20GB. The vendor recommends 32GB or more of unified memory (Apple Silicon) or 32GB or more of VRAM. A quantized GGUF build runs from a minimum of 18GB of system memory. Thinking Machines Lab released it as open weights on November 20, 2026, under Apache 2.0 (with the company's Model Acceptable Use Policy). Its defining feature is looping: the middle 12 layers run three times, so although only 26.4B of weights are stored, each token passes through the equivalent of about 5.4B weights. Memory is set by total parameters; compute is set by the loop-inclusive active parameters. This article covers a requirements table, what looping means, running on Mac, NVIDIA GPUs and CPU, how to use it with LM Studio, llama.cpp, Ollama and vLLM, and how it compares with Inkling 2 Small and similar-size models.
Information is as of November 21, 2026, from primary sources (the Thinking Machines Lab announcement, the Hugging Face model card and config.json, the LM Studio model page, and unsloth's GGUF page). Values given by the vendor are labeled "official"; values we computed from parameter counts are labeled "estimate". All benchmarks are vendor-reported (Thinking Machines Lab), not third-party reproductions.
Requirements at a glance
| Format | Weight size | With 8K context (rough) | With 32K context (rough) | Typical hardware |
|---|---|---|---|---|
| BF16 (theoretical, 26.4B x 2 bytes; estimate) | ~52.8GB | ~54GB | ~55GB | 64GB+ memory or VRAM |
| Official checkpoint (mixed NVFP4 + MXFP8; official) | ~19.5GB (official: ~20GB) | ~21GB | ~22GB | 32GB Mac, 32GB VRAM GPU, DGX Spark |
| GGUF UD-Q8_K_XL (unsloth) | 22.87GB | ~24GB | ~25GB | 32GB Mac, 32GB VRAM |
| GGUF UD-Q4_K_XL (unsloth) | 17.63GB | ~19GB | ~20GB | 24GB VRAM, 32GB Mac |
| GGUF UD-Q4_K_M (unsloth) | 16.61GB | ~18GB | ~19GB | 24GB VRAM, 32GB Mac (LM Studio minimum is 18GB) |
| GGUF UD-Q2_K_XL (unsloth) | 9.14GB | ~11GB | ~11GB | 16GB VRAM, 16GB Mac (lower accuracy) |
| GGUF UD-IQ1_S (unsloth) | 7.76GB | ~9GB | ~10GB | 12GB VRAM, 16GB Mac (much lower accuracy) |
For weight sizes, the official checkpoint and GGUF figures are the file sizes shown on Hugging Face, while BF16 is an estimate (parameters x bytes). The "with context" columns add KV cache (an estimate, using the conservative figure explained below) and about 1GB of working space; actual use varies with runtime and settings. BF16 is not officially distributed. The official checkpoint is about 20GB rather than 26.4B x 4 bits = 13.2GB, most likely because the last four layers are 8-bit (MXFP8) and components such as the vision encoder and embeddings are not 4-bit (the vendor does not spell out this breakdown).
If you want to check quickly whether it runs on your own GPU or Mac, the VRAM calculator (free, no sign-up) estimates required VRAM from the model, quantization and context length you select. Inkling 2 Mini is available in its model list.
Reading the official recommendation directly gives three conditions: one NVIDIA DGX Spark; an Apple Silicon Mac (MacBook, Mac mini or Mac Studio) with 32GB or more of unified memory; or another GPU/platform with 32GB or more of VRAM. LM Studio's page gives "at least 18GB of system memory" as the floor for the GGUF and notes that this floor leaves no room for a full 1M-token context.
What is Inkling 2 Mini? What was released
After the original Inkling in July 2026 and Inkling 2 Small on November 5, Thinking Machines Lab released Inkling 2 Mini on November 20, 2026. The announcement describes it as a fast model "small enough to run on your laptop" and as the company's first released looped transformer. According to the model card, it takes text and images as input and outputs text only; images are best between 40 and 4096 pixels per side. Weights are on Hugging Face, and it can also be used for inference and customization on Tinker. Two variants are published: the standard model and a research model with a variable number of loops.
| Item | Details (source) |
|---|---|
| Provider | Thinking Machines Lab |
| Release date | November 20, 2026 |
| Total parameters | 26.4B |
| Active parameters | 3.4B per token (one pass) |
| Effective active parameters | ~5.4B (each extra pass adds ~1.0B; three passes give 5.4B) |
| Context length | Up to 1M tokens (model card; config says 1,048,576) |
| Input / output | Input: text and images; output: text |
| License | Apache 2.0 (plus Model Acceptable Use Policy) |
| Precision | Mixed NVFP4 and MXFP8 (official checkpoint ~20GB) |
| Other files | Variable-loop research model; MTP head file (~636MB) |
A looped MoE: why 3.4B becomes ~5.4B
Inkling 2 Mini stores 24 layers in four blocks of six. The second and third blocks (the middle 12 layers) run three times with the same weights; the first and last blocks run once. That makes 12 + 12 x 3 = 48 layer applications per token. Only 24 layers of weights are stored, yet the computation approaches that of 48 layers. The model card says each extra pass adds about 1.0B effective active parameters, giving about 5.4B with three passes.
- Stored weights: 26.4B (this sets memory and file size)
- One-pass active: 3.4B (the amount touched in one pass of the MoE, which selects 8 of 128 experts)
- Loop-inclusive effective active: ~5.4B (a guide to per-token compute)
- Experts: 8 of 128 per layer, no shared experts (the announcement says pretraining started at 8 of 384 and was pruned mid-training to the 128 most-used experts)
- Attention: 20 local layers (1,024-token window) and 4 global layers (config.json); the last layer of each block is global
According to the announcement, at an intermediate training step a model without loops needed 1.51-1.91x the compute to match the middle-loop model's loss. A non-looped model also used 7.5% less training compute but needed 140% more inference compute to reach the same loss. These are vendor experiments. The practical meaning is a design that keeps memory small while increasing the amount of "thinking" per token. The model card also lists, as limitations, that per-token compute is higher than the active-parameter count suggests and that the minimum reasoning effort does not fully disable reasoning.
A general MoE caveat applies here too. Even with few active parameters, which experts get selected is unknown in advance, so all weights must be held in memory. Think of speed as tied to the active side and required memory as tied to the total. Because this model rereads the same weights three times, memory bandwidth feeds directly into speed. The announcement says its theoretical speed ceiling was computed at 614GB/s bandwidth, BF16 weights, batch size one, including the repeated reads across loops, but it gives no measured tokens per second.
KV cache estimate: it may grow because of the loops
Per config.json, there are 8 KV heads with head dim 128, so in BF16 each token and layer takes 2 x 8 x 128 x 2 bytes = 4,096 bytes. Of the 24 layers, 20 use a 1,024-token sliding window and 4 are global, so only the four global layers hold KV for the full context. The following are our estimates, not official figures. We could not confirm in primary sources whether each loop pass keeps its own KV, so we show two cases.
| Context length | KV (shared across passes, minimum estimate) | KV (separate per pass, conservative estimate) |
|---|---|---|
| 8,192 | ~0.22GB | ~0.44GB |
| 32,768 | ~0.62GB | ~1.24GB |
| 131,072 | ~2.2GB | ~4.5GB |
| 1,048,576 (1M) | ~17.3GB | ~34.5GB |
The minimum estimate is "4 global layers x 4,096 bytes x tokens + 20 local layers x 4,096 bytes x 1,024". The conservative estimate assumes the middle-block layers keep separate KV for each of three passes, counting 8 global and 40 local applications. The "with context" columns in the table use the conservative values. Using the full 1M tokens would add an estimated 17-35GB of KV on top of the official 20GB, which is unrealistic on a 32GB machine. In practice, aim for roughly 32K-128K.
Running on Apple Silicon (Mac)
The official recommendation is an Apple Silicon Mac with 32GB or more of unified memory. The announcement's demo is a MacBook Pro (M5 Pro, 48GB) running offline. The same page gives "any Mac with at least 32GB RAM" as the minimum for running with Ollama. The model card lists Ollama and Exo Labs as runtimes for Apple Silicon.
- 16GB Mac: the smallest GGUF builds (UD-IQ1_S 7.76GB, UD-Q2_K_XL 9.14GB) fit as weights (estimate), but accuracy drops sharply and this is outside the official recommendation
- 24GB Mac: UD-Q4_K_M (16.61GB) fits as weights, but once macOS and the KV for your context are added, headroom is thin (estimate)
- 32GB+ Mac: the official floor. The official-checkpoint equivalent (~20GB) or UD-Q4_K_XL (17.63GB) fits with around 32K of context by our calculation
- 48GB+ Mac: comfortable room for longer context (~128K) or UD-Q8_K_XL (22.87GB) (estimate)
On a Mac, the GPU can use only a limited share of unified memory, not all of it. The Mac guidance above is a rough headroom check after accounting for that limit and has not been verified on real hardware. LM Studio's page says the 17.50GB GGUF runs with a minimum of 18GB of system memory.
Running on NVIDIA GPUs
The official recommendation is one DGX Spark or 32GB or more of VRAM. The announcement says DGX Spark has native NVFP4 support, and in a Stanford Intelligence per Watt comparison on DGX Spark (128GB), Inkling 2 Mini (MXFP8, low reasoning effort) scored 0.703 versus 0.430 for Qwen3.6-35B-A3B and 0.159 for Gemma-4-26B-A4B (as reported via the vendor).
- 32GB+ VRAM: holds the official checkpoint (~20GB of weights) with around 32K of context by our calculation; the officially recommended setup with vLLM or SGLang
- 24GB VRAM (RTX 3090/4090 class): GGUF UD-Q4_K_M (16.61GB) or UD-Q4_K_XL (17.63GB) fits as weights; keep context short (estimate)
- 16GB VRAM: UD-IQ3_XXS (11.66GB) or UD-Q3_K_XL (13.63GB) fits as weights, trading accuracy (estimate)
- 12GB or less: UD-Q2_K_XL (9.14GB) or smaller; usefulness depends on the task, outside the official recommendation
Because this is an MoE, one option when it does not fit in VRAM is to keep some experts in CPU memory with a runtime such as llama.cpp. Note, however, that the loop structure is new, and the model card states that local runtime support for looped models is new and speed varies by runtime and hardware. Measure speed on your own setup before committing.
CPU-only execution
Even without a GPU, LM Studio's page gives 18GB of system memory as the floor for the GGUF. As an MoE with few active parameters, it should be easier to run than a dense model of the same weight, but rereading the same weights three times makes memory bandwidth the likely bottleneck. We found no measured CPU tokens per second in official material or in the third-party information we could confirm. If you expect interactive speeds, assume a wide-bandwidth DDR5 setup or a GPU assist, and start with a short context.
How to use it: LM Studio, llama.cpp, Ollama, vLLM, transformers
The model card lists Hugging Face Transformers, SGLang, vLLM, TokenSpeed, NVIDIA Dynamo, Unsloth, llama.cpp, Llama App, Ollama and Exo Labs as inference frameworks. The commands below come from the model card and unsloth's GGUF page and have not been run on real hardware. The easiest route is LM Studio: download the GGUF build (17.50GB) from the model page (lmstudio.ai/models/inkling-2-mini). The page lists no dedicated command; you get it with the "Get" button.
llama.cpp (GGUF). The procedure on unsloth's page, using UD-Q4_K_M as the example; swap the suffix for other quantizations.
curl -LsSf https://llama.app/install.sh | sh
llama serve -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_M
llama cli -hf unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_MOllama. Also from unsloth's page.
ollama run hf.co/unsloth/Inkling-2-Mini-GGUF:UD-Q4_K_MvLLM and SGLang (official NVFP4 checkpoint). From the model card; both start an OpenAI-compatible API (vLLM at http://localhost:8000/v1/chat/completions, SGLang at http://localhost:30000/v1/chat/completions).
# vLLM
pip install vllm
vllm serve "thinkingmachines/Inkling-2-Mini"
# SGLang
pip install sglang
python3 -m sglang.launch_server --model-path "thinkingmachines/Inkling-2-Mini" --host 0.0.0.0 --port 30000
# Docker Model Runner
docker model run hf.co/thinkingmachines/Inkling-2-MiniTransformers. The model card snippet uses the pipeline that supports image input (the page also shows a load example using AutoModelForSeq2SeqLM; that class name is as displayed on the page, and the pipeline is the safer choice for ordinary chat use).
# pip install -U transformers accelerate
from transformers import pipeline
pipe = pipeline("image-text-to-text", model="thinkingmachines/Inkling-2-Mini")Reasoning effort is adjustable, but even the minimum does not fully switch reasoning off. The model card warns it can get stuck in repetitive reasoning and recommends setting limits on output tokens and agent turns. The Tinker playground (tinker.thinkingmachines.ai/playground) lets you try inference without obtaining the weights.
Comparison with Inkling 2 Small, the original Inkling and similar-size models
| Item | Inkling 2 Mini | Inkling 2 Small | Inkling (original) |
|---|---|---|---|
| Total parameters | 26.4B | 594B | 975B |
| Active parameters | 3.4B (~5.4B effective) | 17B | ~41B |
| Structure | Looped MoE | MoE | MoE |
| Context length | Up to 1M | (announcement gives no single maximum) | Up to 1M |
| Input | Text, image | Text, image | Text, image, audio |
| Official distribution size | ~20GB | ~393.5GB in NVFP4 | (see separate article) |
| Memory guide (official) | 32GB+ | 500GB+ (NVFP4) | see separate article |
| License | Apache 2.0 | Apache 2.0 | Apache 2.0 |
For Inkling 2 Small, see our Inkling 2 Small article; for the original Inkling's memory needs, see our Inkling requirements guide. The model card says Mini is weaker than Small on knowledge (factual recall) and more prone to confident errors. On long-horizon agentic work, however, it shows a similar tendency, and the announcement says it approaches Small on PMPP-Hard. In short, it lets you try the same line of thinking locally without data-center GPUs.
For similar-size comparisons, the announcement's table lists Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. Below are the vendor-reported values for items that show the gap with Qwen3.8-27B.
| Benchmark | Inkling 2 Mini | Qwen3.8-27B |
|---|---|---|
| SWE-bench Verified | 79.2% | 77.8% |
| DeepSWE v1.1 | 48.2% | 42.2% |
| SWE-bench Pro v2 (Hard) | 77.9% | 73.8% |
| Terminal Bench v2.1 | 71.6% | 73.0% |
| BrowseComp (with compaction) | 79.0% | 61.6% |
| MCP Atlas (public) | 82.7% | 77.3% |
| OmniDocBench v1.6 | 92.8% | 91.5% |
| MMMU Pro (Standard 10) | 74.9% | 76.3% |
| LongBench v2 | 66.9% | 52.3% |
| HLE-Diamond | 15.2% | 15.3% |
A few cautions. First, these are self-reported values, and evaluation settings (harness versions, reasoning effort) affect the results. Second, Qwen3.8-27B leads on many rows of the comparison table (Terminal Bench v2.1, AA-Briefcase, GDPVal-AA, MMMU Pro and others). Third, the model card admits that multi-turn chat and many-constraint instructions are not strengths (MultiChallenge 50.0%, ComplexConstraints v1.1 33.4%). A fair reading is that it is strong for coding, search and tool-use agent work, while chat and factual recall leave room to consider other models. We give no memory-requirement comparison with Gemma 4 31B and others here because we have no like-for-like primary-source figures.
License and commercial-use notes
The license is Apache 2.0, which permits commercial use, modification and redistribution. The model card also states that Thinking Machines' Model Acceptable Use Policy applies. When redistributing or embedding, keep the license text and copyright notice. The model card also lists limitations such as hallucination, missed detailed instructions and lower quality outside English (in Chinese testing, it cites clipped phrasing and mixed languages). We could not find an official Japanese benchmark, so if you use it in Japanese, check quality on your own data first.
The agentic caveats matter most. The model card says it may take destructive actions such as deleting or overwriting files without confirmation, and may misreport outcomes. Run it in a sandbox and require confirmation for destructive actions. Do not use it for medical, legal or safety-critical decisions without validation.
Troubleshooting (out of memory, won't run)
- Out of memory at startup: use a lower quantization (UD-Q4_K_XL -> UD-Q4_K_M -> UD-Q3_K_XL), or cut the context to 8K-32K. Leaving it at 1M tokens needs tens of GB for KV alone (estimate)
- Slow: this model rereads the same weights three times because of the loops, so low-bandwidth systems tend to be slow. Runtime support for looping is new and speed varies by implementation (model card). Update your runtime to the latest version
- Runtime won't load it: update llama.cpp, Ollama or LM Studio to the latest version; older builds may not support the loop structure or newer quantizations
- Answers repeat the same reasoning: a known behavior in the model card. Set limits on output tokens and agent turns
- Mixed or unnatural language in Japanese: the model card notes lower quality outside English. State the output language in the prompt and validate on your own data for important uses
- Images don't work: images are recommended at 40-4096 pixels per side. For GGUF, image input may need a separate projector file (not confirmed in this article)
FAQ
How much memory does Inkling 2 Mini need?
The official checkpoint is about 20GB, and the vendor recommends 32GB or more of unified memory (Apple Silicon) or 32GB or more of VRAM. For the GGUF build, LM Studio's page gives a minimum of 18GB of system memory. The long 1M-token context does not fit at that floor.
Why is it 26.4B total and 3.4B active, yet also described as 5.4B?
Because the middle 12 layers run three times with the same weights. Each extra pass adds about 1.0B effective active parameters, giving about 5.4B with three passes (model card). Memory follows the stored 26.4B; compute follows the loop-inclusive ~5.4B.
Does it run on a 16GB Mac or a 16GB VRAM GPU?
It is outside the official recommendation, but low-bit GGUF builds fit as weights by calculation (UD-Q3_K_XL 13.63GB, UD-Q2_K_XL 9.14GB and so on). After subtracting the OS and KV cache, headroom is small and accuracy drops a lot. 24GB or more is more practical, and the official recommendation is 32GB or more.
Can I use it commercially?
The license is Apache 2.0, which permits commercial use, modification and redistribution. Thinking Machines' Model Acceptable Use Policy also applies. Keep the license text with any redistribution.
How does it differ from Inkling 2 Small?
Small is 594B total and 17B active, and even in NVFP4 it is about 393.5GB (500GB+ of VRAM needed). Mini is a looped model with 26.4B total and 3.4B active (~5.4B effective), at about 20GB. Mini is weaker on factual recall, but the announcement says it shows a similar tendency to Small on long-horizon agentic work.
Can I trust the benchmark results?
All numbers here are vendor-reported by Thinking Machines Lab, not third-party reproductions. Evaluation settings affect results, and Qwen3.8-27B leads on some rows. Judge fit by testing on your own tasks.
株式会社オブライト
Oflight Inc. is a Tokyo-based IT company providing software development, digital signage, web development, network systems, and IoT/AR/VR technology services.
Related free tools (no sign-up, instant results)
Feel free to contact us
Contact Us