Articles tagged "MoE"

Moonshot released Kimi K3 open weights on July 26, 2026. At MXFP4 the weights alone are ~1.4TB — self-hosting means a multi-node cluster, not one GPU.

LongCat-2.0 is Meituan's 1.6T-parameter, MIT-licensed MoE model. Running it locally needs roughly 3,800GB in BF16 or about 970GB even at INT4 — no personal PC can hold it. Requirements tables and API pricing ($0.75/$2.95 per 1M tokens), updated July 2026.

DeepSeek V4-Flash needs roughly 160GB at 4-bit and V4-Pro about 920GB. Quick-lookup VRAM, RAM and GPU tables by quantization for both MIT-licensed open-weight MoE models, plus official API pricing and the deepseek-chat / deepseek-reasoner retirement on July 24, 2026. Updated July 2026.

Nemotron 3 Nano runs in about 18GB at 4-bit, Super needs 8x H100-80GB, Ultra needs 4x B200 at NVFP4. VRAM, GPU and quantization tables for NVIDIA's open-weight MoE family. Updated July 2026.

GLM-5.2 needs roughly 430GB at 4-bit and about 1.5TB at BF16 in combined memory. Quick-lookup VRAM, RAM and quantization tables for running Z.ai's 753B / ~40B-active open-weight MoE (MIT-licensed) locally. Updated July 2026.

Inkling needs from ~280GB (1-bit) to ~600GB (4-bit) combined memory, and 1.9TB at BF16. Quick-lookup VRAM, RAM, disk and quantization tables for running this 975B / 41B-active open-weight MoE locally. Updated July 2026.

Kimi K3 is a 2.8-trillion-parameter open-weight MoE (weights due July 27, 2026), ranked #1 on the frontend-code arena. API pricing is $3 input / $15 output per million tokens. Local runs are estimated at 650GB–1TB, needing server-class hardware. How it differs from K2.

**Google DeepMind's Gemma Team released the Gemma 4 Technical Report as [arXiv:2607.02770](https://arxiv.org/abs/2607.02770) on 2026-07-02**. The paper introduces **2.3B / 12B / 31B parameter models**, **both Dense and MoE variants**, **natively multimodal (text / image / audio)**, a **12B encoder-free unified design** (raw audio and image patches processed directly without separate encoders), a **built-in reasoning (thinking) mode**, **improved vision / audio encoders**, **architectural refinements for inference speed, memory efficiency, and long context**, and **competitive performance against larger open models on STEM, multimodal, and long-context benchmarks**. Over 300 authors contributed. Open weights allow commercial use, distributed via Hugging Face and Ollama. Sits alongside [Qwen 3.6-35B-A3B](../columns/qwen36-35b-a3b-uncensored-abliterated-2026-07) and the [Local LLM June 2026 update](../columns/local-llm-landscape-2026-june-update) as a new chapter at the open-weights frontier. **A milestone in Google's open-weights strategy**; the encoder-free unified design departs from Qwen / Llama / DeepSeek multimodality (separate vision encoder + projection). The **reasoning mode** mirrors the extended-thinking modes of Anthropic and OpenAI closed models — the open ecosystem catching up. Caveats: commercial-license fine print, potential systemic-risk classification (EU AI Act's 10^25 FLOPs threshold), and heavy Google Cloud Vertex AI integration bias.

**Qwen3.6-35B-A3B-Uncensored / Abliterated** is a family of **community-produced derivatives of Alibaba's Qwen 3.6-35B-A3B** (a 35B MoE with 3B active parameters, 262K context, and hybrid attention) with **refusal behaviors surgically removed** ([HackerNoon overview](https://hackernoon.com/qwen36-35b-a3b-uncensored-a-35b-moe-model-with-262k-context) / [HauhauCS Aggressive](https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive) / [huihui-ai abliterated](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated) / [wangzhang abliterated](https://huggingface.co/wangzhang/Qwen3.6-35B-A3B-abliterated) / [prithivMLmods Aggressive](https://huggingface.co/prithivMLmods/Qwen3.6-35B-A3B-Uncensored-Aggressive)). **Base model specs**: **35B total / 3B active parameters** (MoE, sparse experts), **40 layers**, **hybrid attention** in a **3:1 ratio** (linear + full softmax), **native 262K-token context**, and **native text / image / video** multimodal input. Alibaba positions it as a flagship of its open-weights strategy. **The abliteration technique**: **the refusal direction is removed with LoRA-based steering** on attention and MLP projections. It layers on **Expert-Granular Abliteration (EGA)** (abliterating per-expert `down_proj` slices per layer) and **MoE router suppression** (deactivating safety experts at the router stage) — techniques adapted for the MoE architecture. HauhauCS reports **0 refusals across 465 test prompts**. The philosophy: preserve 100% of the base Qwen 3.6-35B's capability, remove only refusal. **Available variants**: - **HauhauCS-Aggressive** ([HF](https://huggingface.co/HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive) / [Ollama](https://ollama.com/fredrezones55/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive)): the most aggressive refusal removal - **huihui-ai Huihui-Qwen3.6-abliterated** ([HF](https://huggingface.co/huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated) / [Ollama](https://ollama.com/huihui_ai/Qwen3.6-abliterated)): from the established huihui-ai team - **wangzhang abliterated** ([HF](https://huggingface.co/wangzhang/Qwen3.6-35B-A3B-abliterated)) - **prithivMLmods Uncensored-Aggressive** ([HF](https://huggingface.co/prithivMLmods/Qwen3.6-35B-A3B-Uncensored-Aggressive)) Each ships with **quantization variants** (GGUF Q4 / Q5 / Q8 / FP16), spanning consumer GPUs (RTX 5090 32GB) up to H100 servers. **Ethical and legal considerations**: abliterated models **can produce content that Qwen would normally refuse** (illegal drugs, offensive-security code, dangerous-material synthesis, etc.). Legitimate research, jailbreak-resistance testing, roleplay, and adult-content use cases exist, but **enterprise and commercial adoption carries meaningful legal risk**. Compliance with the EU AI Act (effective August 2026) and Japanese PPC guidance is also in question. **The responsibility sits entirely with the user**; Alibaba's Qwen team is not involved. **Positioning**: alongside our [Local LLM June 2026 Update](../columns/local-llm-landscape-2026-june-update), [Kimi K2.7-Code](../columns/kimi-k2-7-code-moonshot-ai-2026-06), and [Ornith-1.0](../columns/ornith-1-0-deepreinforce-agentic-coding-2026-06), this is a case study showing that **"safety-stripping techniques" scale into the MoE era** in the open-weights ecosystem.

A primary-source deep dive on **Kimi K2.7-Code**, released June 12, 2026 by Moonshot AI (Beijing). Grounded in the [Hugging Face model card](https://huggingface.co/moonshotai/Kimi-K2.7-Code), [MarkTechPost](https://www.marktechpost.com/2026/06/12/moonshot-ai-releases-kimi-k2-7-code-a-coding-model-reporting-21-8-on-kimi-code-bench-v2-over-k2-6/), and [VentureBeat's skepticism piece](https://venturebeat.com/technology/kimi-k2-7-code-cuts-thinking-tokens-30-practitioners-say-benchmarks-dont-check-out). Covers the 1T-total / 32B-active MoE architecture (384 experts, 8 routed + 1 shared), 256K context, MoonViT ~400M vision encoder, native INT4, forced-on thinking mode. License is **Modified MIT** (attribution required only above 100M MAU or $20M MRR), API pricing is $0.95 input / $0.19 cache-hit / $4.00 output per 1M tokens — roughly **1/18 of Claude Opus 4.8's output price**. OpenAI + Anthropic-compatible endpoints drop straight into Claude Code / Cursor / Aider / Cline / cmux. Moonshot self-reports **+21.8% vs K2.6 on its own Kimi Code Bench v2 and -30% reasoning tokens**, but **all public benchmarks are Moonshot's own proprietary suites; independent SWE-bench Verified / Pro / FrontierCode scores are not yet available as of June 15, 2026** (VentureBeat). For Japanese enterprises the column flags two critical caveats: **(1) both `api.moonshot.cn` and the Singapore-subsidiary-run `api.moonshot.ai` remain exposed to PRC National Intelligence Law Article 7 compelled disclosure (set against Japan's PPC DeepSeek alert of February 3, 2025 and the Digital Agency notice of February 6, 2025), and (2) the only reliable mitigation is Hugging Face self-hosting (~4-8 H100, ~595GB INT4) following the Mizuho / Lion Qwen-on-domestic-infrastructure precedent**.

Overview of DeepSeek V4 Preview, released on April 24, 2026: two open-weight Mixture-of-Experts variants (V4-Pro at 1.6T total / 49B active and V4-Flash at 284B / 13B), 1-million-token context, weights on Hugging Face, and rollout via API and chat — based on official information.

GLM-5.1 by Z.ai (released April 7, 2026) is the first open-source LLM to top SWE-bench Pro at 58.4%, surpassing GPT-5.4 (57.7%) and Claude Opus 4.6 (57.3%). This guide covers its 744B/40B-active MoE architecture, MIT license, 8-hour autonomous task capability, and setup via Ollama.

Kimi K2.5, released by Moonshot AI on January 27, 2026, is a 1 trillion parameter (32B active) MoE model under the MIT License. It scores 76.8% on SWE-bench, 99.0% on HumanEval, and 87.6% on GPQA Diamond. This guide covers its architecture, hardware requirements, Ollama setup, and practical use cases.

Mistral Small 4, released March 2026, unifies reasoning, multimodal vision, and agentic coding in a 119B MoE model under Apache 2.0. Supports 11 languages including Japanese. Full specs, setup guide, and model comparisons.

MiniMax M2.5 achieves 80.2% on SWE-bench Verified using proprietary Lightning Attention in a 230B MoE model. Full breakdown of architecture, benchmarks, license terms, and setup instructions.

A comprehensive analysis of Rakuten AI 3.0's Mixture of Experts architecture with 700B parameters. Explore the 8-expert configuration, 40B active parameter efficiency, and technical background behind achieving 8.88 on Japanese MT-Bench.

A technical deep dive into NemoClaw's NIM inference microservices and Nemotron model family. We examine containerized API endpoints, elastic scaling, Nemotron 3 Super performance (120B parameters, MoE with 12B active), deployment comparisons across AWS, Azure, GCP, and on-premises, lightweight edge device operations, and partner integration use cases with Salesforce, CrowdStrike, and more.