Kimi K3 Weights: Self-Host GPU Specs & Cost (2026)

Kimi K3's weights are public — and you still cannot run them on your own GPU or Mac. At native MXFP4 (4-bit), 2.8 trillion parameters come to roughly 1.4TB of weights alone; add KV cache and activation headroom and self-hosting means dozens of 80GB-class GPUs across multiple nodes. This is the size band where "open weights" stops implying "runnable locally," leaving three realistic paths: the official API, a day-0 hosting provider, or your own cluster. Here is what shipped, the hardware math, the inference-stack constraints, and where smaller models remain the better answer.

The model itself — architecture, benchmarks, API pricing, and how it differs from K2 — is covered in Kimi K3 explained: 2.8T MoE specs, pricing and local requirements. This is the follow-up now that weights are actually out.

Moonshot AI released the Kimi K3 weights free of charge on July 26, 2026 (US Eastern evening; the morning of July 27 in Japan) — ahead of the July 27 target the company had communicated. That put weights roughly ten days behind the API, which launched on July 16. Together AI and Modal announced day-0 hosted access timed to the drop, so there was a way to call the model from hour one without downloading anything.

One detail matters more than it first appears: MXFP4 here is not post-training quantization but quantization-aware training applied from the SFT stage. The 1.4TB artifact is the native distribution, not a lossy convenience build, so "just quantize it further" is not a straightforward move. Reported download sizes varied in the first hours after release — most likely differences in how sharded files are counted — so check the file listing on the distribution page before pulling.

1.4TB is a floor, not a working figure. Three things get added on top:

- Block scales and metadata: MXFP4 carries per-block scaling factors, so real size exceeds a naive 4-bits-times-parameters calculation
- KV cache: with a 1M-token context, this dominates once you actually push long inputs, and it scales with concurrency
- Activations and runtime buffers: MXFP8 activations, communication buffers, and fragmentation headroom

Moonshot's own guidance for distributed inference is a 64-plus accelerator supernode combining expert parallelism with tensor parallelism. Having 1.4TB of aggregate memory is not sufficient on its own: you also need interconnect bandwidth (InfiniBand/NVLink class) that survives routing across 896 experts. That is the decisive difference from hundred-billion-class open models like DeepSeek V4 or GLM-5.2.

The hopeful reading — "it's MoE, only ~50B is active, so surely it fits" — does not hold. In an MoE model, sparsity applies to compute, not to memory. You cannot know in advance which expert a token will route to, so all 896 experts must sit somewhere immediately addressable. Offloading idle experts to disk is technically possible and practically useless: you would pay SSD read latency on every token.

Even an M3 Ultra with 512GB of unified memory falls well short of 1.4TB. If your goal is running an LLM on hardware you own, K3 is simply not the model — a 7B–70B model via GGUF quantization fits the goal far better. Size the VRAM with the VRAM calculator.

K3 uses a new architecture (KDA plus Attention Residuals), which means a runtime that can read the weights is not automatically a runtime that serves K3 correctly or quickly. Moonshot contributed a KDA implementation with prefill caching to the vLLM community and shipped it alongside the weights.

- vLLM: the primary target, since the upstream contribution went here. Assumes MoE-aware scheduling and the prefill cache
- SGLang: proven on MoE workloads and a reasonable candidate for expert-parallel deployments
- TensorRT-LLM: an option when squeezing NVIDIA-specific performance, at the cost of a heavier build process
- llama.cpp / Ollama / LM Studio: designed around single-machine inference and out of scope at this scale

If you are standing this up, start by confirming your inference engine version actually supports KDA. On a pre-KDA build you will either get broken output quality or a hard failure in routing.

The real significance of the weight release is not "anyone can run it at home" — it is that a path exists to run it without sending data to anyone else. For organisations where routing data through a Chinese-hosted API is a blocker, the existence of that path is itself the value. Executing on it, though, takes eight-node-class infrastructure and the team to operate it, which narrows the field considerably.

The same holds on rented cloud GPUs. Against the hourly cost of keeping 64 H100 80GBs reserved, the API wins unless load is constant. Self-hosting pencils out only at high token volume with high cluster utilisation — that is the honest conclusion.

- Read the actual license: "modified MIT" is the reporting; the shipped LICENSE governs commercial use, redistribution, and output handling
- Benchmarks are vendor-published: GPQA-Diamond 93.5, SWE Marathon 42.0 and similar come from the developer. Evaluate on your own tasks separately
- Price the 1M context: KV cache and latency scale with input length. Measure at your real prompt sizes
- Early figures move: sizes, recommended topologies and runtime support get revised as community verification lands

The K3 weight release is a milestone: the largest open-weight model to date is now downloadable by anyone. But actually serving that 1.4TB is limited to organisations that can stand up eight-node-class infrastructure, and for most developers the practical entry point remains the API or a hosting provider.

If you are shopping for a model to run locally, pick one that fits the VRAM you have — that path reaches the goal, and K3 does not. K3's value is the option it preserves: when data cannot leave your perimeter, a route exists. Figures and recommended topologies are still moving this soon after release, so check the distribution page and your inference engine's release notes directly before committing.