What Is llama-modchip? Run DeepSeek V4 Flash 284B on 64GB RA

llama-modchip (GitHub: synchopate/llama-modchip) is an Apache-2.0 open-source bundle of kernel patches for ggml/llama.cpp, quantization transcoders, and runtime tools. Its headline claim is that it runs DeepSeek V4 Flash, a Mixture-of-Experts (MoE) model with 284.33B total and 13.17B active parameters, on a PC with 64GB of RAM and no GPU, including a 256K-token context. According to the author, a Ryzen 9 9955HX (16 cores) with 64GB and no GPU reaches about 43-44 tok/s for prefill (prompt processing), about 9-10 tok/s for plain decoding, and about 11.5-14.7 tok/s with self-speculative decoding. On November 9, 2026 the Hacker News post "Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU" reached 423 points.

This article follows the README (v2.4.0) as of November 2026 and covers what it does, the memory requirements, how it works, how to install it, how it differs from upstream llama.cpp, Ollama, LM Studio, and ktransformers, and where it is a poor fit. Every speed, size, and quality number below comes from the README as reported by the author and community. We have not verified them ourselves.

- What it is: a llama.cpp-based fork focused on CPU inference, built to run MoE models entirely from system RAM
- What runs: DeepSeek V4 Flash 284B (requantized to 56.56GiB), Nemotron-3.5-30B-A3B, Qwen3-30B-A3B, Qwen3-14B, Nemotron-H-8B-128K
- Catch: it uses a custom quantization format (Fractal-64), so stock llama.cpp, Ollama, and LM Studio cannot load these models

The README highlights three things.

- Fit a 284B model into 64GB of RAM: the Unsloth UD-IQ1_S build of DeepSeek V4 Flash is 76.87GiB (82.5GB), more than the roughly 62.5GiB usable on a 64GB Linux machine. llama-modchip shrinks it to 56.56GiB (1.71 bits per weight effective) and, with a 256K-token KV cache, peaks at 57.61GiB VmRSS with no swap
- CPU kernel optimizations: multi-column tiled prefill, MQA8 FlashAttention, and non-temporal reads of MoE weights so they do not evict the KV cache from L3. The README says prefill rose from about 25 tok/s to about 44 tok/s
- Self-speculative decoding and fast context restore: a roughly 1.64MB n-gram drafter called Stockfish-PVS drafts several tokens ahead without a separate model, and conversation KV slots are snapshotted to /dev/shm and restored in tens of milliseconds (the README gives 13.1-32.6ms)

The bundled ./modchip command includes interactive chat (chat), a web chat UI (web), a quick benchmark (bench), a claude mode that points Claude Code at the local engine, a quality gate (eval), and a quantizer (quantize).

If you want to check quickly whether a model fits your own GPU or Mac, our free VRAM calculator (no sign-up) estimates required VRAM from the model, quantization, and context length you choose. DeepSeek V4-Flash is available in its model list. Note that it assumes standard quantization; llama-modchip's custom 1.71 bpw requantization (about 56.6GiB) is not an option there, so the real figure with llama-modchip will be smaller than the calculator's.

For the model itself and its memory needs under standard quantization, see our DeepSeek V4 requirements and pricing guide.

The 284B numbers in the README were measured on a Ryzen 9 9955HX (Zen 5, 16 cores, 64GB DDR5, no GPU). The default lossless Top-6 setting, which evaluates every routed expert, gives about 9.06-9.95 tok/s, and a Top-5 setting gives about 9.7-9.9 tok/s at roughly +0.42% perplexity. Reducing Top-K is an optional runtime flag that trades a little quality for speed.

For 30B-class models, the README also lists community results: up to 111.62 tok/s on a Threadripper PRO 7985WX (256GB of 8-channel DDR5), about 49-58 tok/s on an Apple M4 Max (64GB), about 19-23 tok/s on a Ryzen 7 8745H (96GB DDR5-5600), and about 13 tok/s on a Ryzen 5 7430U (32GB DDR4). The pattern is that speed depends strongly on core count and memory bandwidth, so desktops and workstations with wide memory do best. Most of these figures are for 30B-class models, not for the 284B DeepSeek V4 Flash on the same machines.

1. Mixed quantization per layer. In DeepSeek V4 Flash, the 256 routed experts take most of the space, so they are mainly stored as IQ1_S (about 1.56 bpw), with sensitive boundary layers kept at IQ2_XXS or IQ1_M. Embeddings, shared experts, and the main MLA projections stay at higher precision such as Q4_K, Q5_K, or F32. Part of the MLA (wq_b and the output projections) uses the custom Fractal-64 format (GGML_TYPE_Q2_0, type ID 40, 64-weight blocks with a BF16 scale). Together this removes 20.31GiB from the 76.87GiB build. For the basics of how memory is spent, see our KV cache and context length guide.

2. MoE loads everything but reads only a slice per token. An MoE model needs memory for all its parameters, but each generated token computes only the active portion (13.17B here). Decode speed is roughly set by how much data is read per token and by memory bandwidth, so this is far lighter than a dense 284B model and makes about 10 tok/s on a CPU realistic. Conversely, if you lack the memory capacity, it simply will not run.

3. Self-speculative decoding. Instead of a separate small draft model, it predicts the next few tokens with an n-gram table built from recent context and verifies them in one pass. The README says acceptance is high in repetitive work such as code refactoring, lifting decode from about 9 tok/s to about 11.5-14.7 tok/s. Free-form prose probably gains less.

On Linux and macOS, the fastest path is to clone the repository and run ./modchip chat, which handles setup and starts a chat.

You can also download a prebuilt Linux x86_64 binary and the GGUF directly from Hugging Face. These are the README's commands. Note that the 284B download is about 56.6GiB.

-t sets the thread count (physical core count is a good starting point) and -c sets the context length. For Windows 10/11 there is a hybrid GPU plus CPU pack (AVVIA_MODCHIP.bat in the windows folder). Once running, use the web UI at http://127.0.0.1:8080 or the OpenAI-compatible server. To check whether your CPU is ready before downloading any model, run the following diagnostic script.

This pipes a script from the internet straight into Python, so read the script at that URL first if you have any doubts.

The key point is compatibility. llama-modchip models use a custom block format, so stock llama.cpp, Ollama, and LM Studio fail with unknown tensor type 40. Conversely, llama-modchip itself supports only a limited set of models. ktransformers is generally known as a separate framework that splits large MoE models across GPU and CPU memory, but we have not tested it for this article, so check each project's documentation for current details. For the wider landscape, see our local LLM inference engine comparison and Ollama vs LM Studio; for another engine that also relies on its own GGUF files, see DwarfStar (ds4).

- All numbers are self-reported: speed, memory, and quality figures come from the author and community, and we found no independent reproduction. Run ./modchip bench on your own machine to confirm
- Quality may be degraded: the routed experts are mostly at about 1.56 bpw (IQ1_S), an extreme low-bit setting, so quality is unlikely to match the full-precision model. The README's "lossless" means identical output before and after the kernel changes on the same checkpoint, not versus the original model
- Compatibility: the custom format cannot be reused with other tools or your existing GGUF files, and updates depend on the project
- Linux-centric: the 284B verification is mainly on Linux. macOS and Windows are documented, but we found no 284B run report for them in the README
- Speed is a floor for interactive use: about 10 tok/s is readable in real time but not suited to long generation or many simultaneous users. With prefill around 44 tok/s, a prompt of tens of thousands of tokens can take ten minutes or more before the first reply
- Little memory headroom: 57.61-58.9GiB measured on a 64GB machine leaves little room for a browser or other apps
- A young personal project: check the license and maintenance status before using it for real work

It suits people who want to try a giant MoE model using only the memory they already own, want snappy 30B-class models on a 64GB desktop or mini PC, or need offline or privacy-bound experiments where cloud APIs are off the table. If quality is the priority, or you need multi-user access or high-volume batch processing, a GPU machine or an API is the more realistic choice.