Articles tagged "Open Weight LLM"

KRAFTON's Raon-1 mini preview is a 28.8B MoE with ~4.3B active. At 4-bit the weights are ~16-20GB and fit a 24GB GPU. Sizes by quantization, setup, benchmarks and comparisons. Updated Oct 2026.

Nanbeige4.3-33B (dense 33B): Q4_K_M is ~20.8GB and fits a 24GB GPU with short context; BF16 ~66GB. GGUF sizes, KV cache, GPU/Mac setups. Released Oct 2026.

Kimi K3.1 Pro requirements: a ~2.8T MoE needs ~1.4TB at 4-bit (est.), so most teams will use the API at $2/$10 per 1M tokens, a third cheaper than K3. Updated Oct 2026.

MiniMax M3.1 Flash (428B MoE, ~23B active): ~250GB weights at 4-bit, 260-300GB RAM. Needs a 512GB Mac or 4 GPUs. Sizes, KV cache, API cost. Updated Oct 2026

Recursive Flint (40B-A2.5B): ~23-26GB weights at 4-bit, 24-30GB memory. 24GB GPU: 3-bit; 12GB: offload; 262K ctx: 48GB. Sizes, tok/s. Updated Oct 2026.

DeepInfer is DeepSeek's Apache 2.0 inference engine (Oct 2026): V4.1-Flash on SM100 GPUs and Ascend 950 only. Requirements, setup, and differences from vLLM.

Nemotron 3.5 Bolt 74B-A9B (24GB): ~22GB GGUF, 24-32GB memory; RTX 5090 or 32GB Mac minimum. Context 128K/256K, tok/s, vs Gemma 4, setup. Updated Oct 2026.

MiniCPM-V 4.7 35B-A3B (Apache-2.0) needs ~20-22GB at 4-bit, 70GB+ in BF16 (est.): RTX 4090/5090 or 32GB Mac. iPhone LES, vLLM/GGUF setup. Updated Oct 2026.

DeepSeek V4.1 Flash (1017) needs ~510GB for the official weights, ~430GB at 4-bit class: 8x H200 or a 512GB Mac. Updated Oct 2026: 763B vs 552B, memory by quant, API pricing.

LFM2.5-24B-A2B needs ~14-15GB at Q4; a 16GB VRAM GPU or 24GB+ Mac runs it. Updated Oct 2026: memory by quantization, GPU/Mac picks, speeds, benchmarks.

Muse Spark Open (816B MoE, 42B active) needs ~2TB at BF16 and 700GB+ at 4-bit. Two M5 Ultra Mac Studios, H200/B200 setups and license terms. Updated Oct 2026.

Fish Audio Drama 3 (Oct 2026): 35B MoE open-weight TTS. No official minimum; 48GB GPU advised. Free commercial use under $500K revenue, API cost, SGLang-Omni.

Holo4-35B-A3B (Apache-2.0) needs ~20GB at 4-bit and ~80GB in BF16 (estimates): RTX 4090/5090 or a 32GB Mac. vLLM and GGUF setup, API cost. Updated Oct 2026.

Mistral Large 4 (1T MoE, 49B active) needs an estimated ~1TB at FP8, ~500-580GB at 4-bit. GPU, Mac and API cost compared. Updated Oct 2026; weights due soon.

Reflection Beam (501B MoE, 23B active) needs an estimated ~501GB at FP8, ~250-290GB at 4-bit. GPU, Mac and CPU-offload fits as of October 2026; weights not out.

Kolibri-1 needs about 78GB in FP8, running on 1x H200 or 2x H100. Aleph Alpha's 78B/3.46B-active Apache 2.0 MoE: VRAM by precision, vLLM launch, German focus. Oct 2026.

Index-Translate needs about 2GB VRAM (2B), 6GB (9B) or 21GB (35B-A3B) at 4-bit. bilibili's 150-language models: GPU, Mac, CPU specs and setup. Oct 2026.

Cloudflare Clef and Clef-flash: open-weight decision models with a Jev-compatible API. Apache 2.0, 41GB minimum VRAM, $0.24/M. Compared with AWS Strands Decider 2B and Jev.

Ternary Bonsai 2 27B (PrismML, Sep 2026): Qwen3.8-27B at 1.76-bit weights, 5.9GB, 16GB RAM or a 24GB GPU, 98.2% of FP16 score. Apache 2.0, updated Sep 2026.

MiMo-V2.6 Flash needs ~170-190GB at 4-bit, ~320-350GB at FP8; Pro ~550-600GB at 4-bit; the 9B distill ~6-7GB. Updated Sep 2026: specs, benchmarks, API prices.

Atria Dawn Preview needs an estimated ~400GB at 4-bit, ~780GB at FP8, or ~1,500GB at BF16. A rundown of this 744B MoE model's specs, vendor-reported benchmarks, API options, and comparison with other open MoE models.

As of Sep 7, 2026, OpenBMB's MiniCPM5-2B (2.52B dense, 131K context, Apache-2.0) needs roughly 1.5GB to 5GB of RAM/VRAM depending on quantization. Here is a GGUF size table, how to run it with Ollama and llama.cpp, and a comparison with Qwen3.5-4B.

As of Sep 2026, Nex-AGI's agentic Nex-N2.5 family (Mini 35B, Pro 397B, Max 1.6T), released Sep 8, needs anywhere from roughly 22GB to over a terabyte of VRAM depending on model and quantization. Here is a per-model, per-quantization sizing table plus GPU and Apple Silicon build guidance.

DeepSeek V4.1-Flash launched Sep 10, 2026: 552B MoE, 8B/16B active params, 890-byte/token KV cache, quarter prior gen, MIT license. Pricing and VRAM specs.

As of Sep 2026, K2 Horizon from MBZUAI's Institute of Foundation Models spans 0.9B to 375B parameters, needing anywhere from under 1GB to roughly 750GB of VRAM depending on quantization. Here is a per-model, per-quantization sizing table plus GPU and Apple Silicon build guidance.

Solar-Open2-250B (15B active MoE): GGUF IQ4_XS ~127GB, Q2_K ~89GB, Q6_K ~191GB. Official spec is 4-8x H200, but a single 96GB GPU works too. As of Sep 2026.

Qwen3.8-Flash-Next needs roughly 75GB (1-bit) to 354GB (BF16) of memory depending on quantization. This 125B-total/6B-active open-weight MoE has been run on a single 24GB RTX 4090 via MoE expert offloading, while the official vLLM/SGLang FP8 recipe needs ~250GB across multiple datacenter GPUs. VRAM and GPU tables inside. Updated August 2026.

Hy4 preview needs about 385GB VRAM at 4-bit and 1.54TB at BF16. Tencent open-weighted this 770B MoE with 1M context in Aug 2026. VRAM tables, GPUs, API cost.

GLM-5.3-Flash needs about 190GB VRAM at 4-bit, 740GB at BF16. VRAM and GPU tables for this 320B/18B open MoE, plus local vs API costs. Updated August 2026.

Ornith 1.5 is an MIT open-weight LLM needing 8GB to 800GB VRAM. Compare GPU picks and quantized VRAM needs for 9B, 35B-A3B and 397B. As of August 2026.

DeepSeek's V4-Flash-Vision-Exp launched Aug 21, 2026 at V4-Flash pricing. On Aug 31, DeepSeek open-sourced the ~168GB FP4/FP8 weights on Hugging Face under MIT. Update covers local hardware needs and when the API still wins.

Unsloth Dynamic 3.0 is a GGUF quant method with fresh imatrix data, up to 10% higher accuracy at same size vs 2.0. Sizes, VRAM, setup. Updated Aug 2026.

GLM-5.3, released Aug 14, 2026, reuses the 743B GLM-5.2 base and gains from post-training alone: a claimed 50% coding jump and top open Terminal-Bench scores.

Qwen3.8-27B weights landed on August 15, 2026. This at-a-glance requirements guide maps VRAM needs to the actual published file sizes: Q4_K_M is 17.1GB and won't fit a 16GB GPU, making IQ4_XS (15.7GB) the practical floor. Licensed Apache-2.0 for commercial use.

DeepSeek V4 Pro hit GA (V4-Pro-0813) Aug 12, 2026 with big benchmark gains. API prices rise Aug 16, output ~2.3x. Covers pricing, self-hosting, Harness v0.1.

DOE's Genesis Open Models Initiative, announced Aug 7, 2026, builds open-weight AI for science. First model: Genesis-Science-1, co-developed with Arcee AI.

Liquid AI's LFM2.5-2.6B is a 2.69B on-device agent model under 2.5GB with 128K context. This guide covers VRAM sizing, benchmarks, throughput, and licensing.

Shieldstral 1.0 3B is Mistral AI's Apache 2.0 moderation model, released Aug 4, 2026. It needs ~16GB VRAM in BF16 and screens both text and images together.

Qwen3.8 Max: Alibaba's 2.4T sparse MoE, ~95B active/token. SWE-bench 87.3%. API from $2.00/$6.00/M tokens (in/out). Open weights promised. Updated Aug. 2026.

MiniMax H3, an open video+audio model, released weights Aug 3, 2026. ComfyUI needs ~42.5GB files, ~24GB VRAM (12GB may work). Covers quantization, GPU sizing.

WASTE is a dependency-free C engine running Kimi K3 (2.78T params) on 29GB RAM by streaming MoE experts from NVMe. Covers real throughput, setup, and limits.

LG AI Research's K-EXAONE 2.0 750B-A37B needs ~1.5TB weights at BF16, ~750GB at FP8, and 375-420GB at 4-bit — hardware math and Apache 2.0 self-host limits.

Moonshot released Kimi K3 open weights on July 26, 2026. At MXFP4 the weights alone are ~1.4TB — self-hosting means a multi-node cluster, not one GPU.

Start with Q4_K_M; step up to Q5_K_M or Q6_K if you have VRAM headroom. This guide explains GGUF naming, the quality/speed/VRAM tradeoffs per level, IQ (imatrix) quants, and how to choose by task. Updated July 2026.

LongCat-2.0 is Meituan's 1.6T-parameter, MIT-licensed MoE model. Running it locally needs roughly 3,800GB in BF16 or about 970GB even at INT4 — no personal PC can hold it. Requirements tables and API pricing ($0.75/$2.95 per 1M tokens), updated July 2026.

DeepSeek V4-Flash needs roughly 160GB at 4-bit and V4-Pro about 920GB. The July 31 V4-Flash-0731 build keeps the same footprint while beating V4-Pro on agent benchmarks. VRAM, RAM and GPU tables by quantization, API pricing, and the legacy model retirement. Updated August 2026.

Nemotron 3 Nano runs in about 18GB at 4-bit, Super needs 8x H100-80GB, Ultra needs 4x B200 at NVFP4. VRAM, GPU and quantization tables for NVIDIA's open-weight MoE family. Updated July 2026.

GLM-5.2 needs roughly 430GB at 4-bit and about 1.5TB at BF16 in combined memory. Quick-lookup VRAM, RAM and quantization tables for running Z.ai's 753B / ~40B-active open-weight MoE (MIT-licensed) locally. Updated July 2026.

Inkling needs from ~280GB (1-bit) to ~600GB (4-bit) combined memory, and 1.9TB at BF16. Quick-lookup VRAM, RAM, disk and quantization tables for running this 975B / 41B-active open-weight MoE locally. Updated July 2026.

Kimi K3 is a 2.8-trillion-parameter open-weight MoE (weights due July 27, 2026), ranked #1 on the frontend-code arena. API pricing is $3 input / $15 output per million tokens. Local runs are estimated at 650GB–1TB, needing server-class hardware. How it differs from K2.

A primary-source deep dive on **Kimi K2.7-Code**, released June 12, 2026 by Moonshot AI (Beijing). Grounded in the [Hugging Face model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code), [MarkTechPost](https://www.marktechpost.com/2026/06/12/moonshot-ai-releases-kimi-k2-7-code-a-coding-model-reporting-21-8-on-kimi-code-bench-v2-over-k2-6/), and [VentureBeat's skepticism piece](https://venturebeat.com/technology/kimi-k2-7-code-cuts-thinking-tokens-30-practitioners-say-benchmarks-dont-check-out). Covers the 1T-total / 32B-active MoE architecture (384 experts, 8 routed + 1 shared), 256K context, MoonViT ~400M vision encoder, native INT4, forced-on thinking mode. License is **Modified MIT** (attribution required only above 100M MAU or $20M MRR), API pricing is $0.95 input / $0.19 cache-hit / $4.00 output per 1M tokens — roughly **1/18 of Claude Opus 4.8's output price**. OpenAI + Anthropic-compatible endpoints drop straight into Claude Code / Cursor / Aider / Cline / cmux. Moonshot self-reports **+21.8% vs K2.6 on its own Kimi Code Bench v2 and -30% reasoning tokens**, but **all public benchmarks are Moonshot's own proprietary suites; independent SWE-bench Verified / Pro / FrontierCode scores are not yet available as of June 15, 2026** (VentureBeat). For Japanese enterprises the column flags two critical caveats: **(1) both `api.moonshot.cn` and the Singapore-subsidiary-run `api.moonshot.ai` remain exposed to PRC National Intelligence Law Article 7 compelled disclosure (set against Japan's PPC DeepSeek alert of February 3, 2025 and the Digital Agency notice of February 6, 2025), and (2) the only reliable mitigation is Hugging Face self-hosting (~4-8 H100, ~595GB INT4) following the Mizuho / Lion Qwen-on-domestic-infrastructure precedent**.