Skip to main content
株式会社オブライト
株式会社オブライト

ggml 1.0.0 Released

API Removals, RPC v9, llama.cpp Upgrade Guide

Published 2026/11/158 min read

ggml 1.0.0 (Nov 13, 2026) removes deprecated APIs and DIAG_MASK ops and moves RPC to v9. RPC users must update every node together. Steps and checklist.

ggml 1.0.0 was released on November 13, 2026 on GitHub (ggml-org/ggml). ggml is the tensor library underneath llama.cpp and whisper.cpp. This release centers on removing deprecated public APIs, moving the RPC protocol to v9, and per-backend optimizations. If you only use llama.cpp or similar tools, updating to the latest version is usually enough. If you run distributed inference over RPC, you must rebuild every node on the same generation and switch them at the same time.

This article reads the official release notes and the related llama.cpp pull requests (#29801 and #31946) to sort out what changed, who needs to do what, the update steps, and a checklist. The official notes contain no performance numbers, so we do not quote any benchmarks.

What is ggml? Three key points

- What it is: a tensor library for machine learning aiming to be simple, portable, and efficient. It is written in C/C++, has no dependencies, and does zero memory allocations at runtime (per the README)
- What 1.0 means: a milestone release that prunes deprecated public APIs and sets the version to 1.0.0 (commit "ggml : bump version to 1.0.0", #1690)
- License: the README states MIT. The GitHub release page itself shows no license

ggml supports 2- to 8-bit integer quantization plus the MXFP4 and NVFP4 microscaling formats, and targets x86, ARM, RISC-V, LoongArch, PowerPC, s390x, and WebAssembly. Backends include CPU, CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU, and Hexagon, and the notes also mention CANN, ZDNN, and OpenVINO.

According to the README, changes to the core library, including the CMake build system, are submitted as pull requests to llama.cpp rather than to the ggml repository. The PR numbers in the release notes, such as llama/29801, are llama.cpp numbers, so ggml reads as a synchronized distribution of work developed in llama.cpp.

What changed in 1.0.0

The release notes list more than 100 commits. The table below covers the ones that matter most to users, based on the official notes and PR descriptions.

AreaChangePRs (llama.cpp)
Public APIRemoval of deprecated APIs and ops (GGML_OP_ADD1, ggml_mul_mat_set_prec, ggml_rope_custom, ggml_upscale_ext, ggml_flash_attn_ext_set_prec)#29801
OpsRemoval of DIAG_MASK_INF / DIAG_MASK_ZERO and four related functions (migrate to TRI)#31946
RPCProtocol v9. Peers whose major version differs are rejected at handshake#29801, #29413, #30415
RoPEAdded ggml_rope_set_truncate / ggml_rope_get_truncate (controls rounding of YaRN correction bounds)#30206, #30802
FP8OCP FP8 support in CUDA; FP8 KV cache support on the llama side#28413, #28392
MoEMetal MoE routing fusion, CUDA NVFP4 gate/up fusion, SYCL device-side MoE scheduler, and more#31615, #31296, #30156
Quantized matmulCUDA few-row MMQ (batch sizes 2-24) and BF16 few-row MMVF#31533, #31959
VulkanFlashAttention prefill kernel for Intel, tuning for Qualcomm#29357, #30829
CoreScheduler skips uploading unused inputs; MoE cache can compute some misses on the CPU#31419, #32100, #30966

Other backend items include Adreno FlashAttention and GEMV kernels in OpenCL, GROUP_NORM, MEAN, COUNT_EQUAL, and PAD_REFLECT_1D in WebGPU, and Q5_0/Q5_1 support plus conv_2d optimization (and conv_3d) in Hexagon. The release notes do not state how much faster any of this is.

The RoPE truncate setting for long-context models

YaRN extends RoPE to handle long contexts, and whether the correction-dimension bounds are rounded to integers slightly changes behavior. 1.0.0 adds ggml_rope_set_truncate and ggml_rope_get_truncate to control that rounding. Per the official notes, the setting is supported on OpenVINO, SYCL, Vulkan, WebGPU, and RPC.

PR #29801 lists YaRN truncate=false support as a remaining follow-up, so it is not safe to assume every path is fully aligned. If you run long-context models, check after updating that your backend supports the setting.

Impact and actions by user type

End users of llama.cpp, Ollama, LM Studio, and similar tools do not touch ggml code, so API removals will not break their code. They only need to apply updates once each tool ships them. We could not confirm from public sources which ggml revision each tool bundles. Ollama and LM Studio pull in changes on their own schedules, so check their release notes. In llama.cpp itself, the deprecated-API removal (#29801) was merged into master on November 12, 2026.

Developers who call ggml directly must replace calls to removed APIs. The description of PR #29801 names what was removed but does not give replacements one by one, so the reliable approach is to search your call sites and compare them with the new ggml.h. Two migrations are confirmed in PR #31946.

- Replace ggml_diag_mask_zero(ctx, a, 0) with ggml_tri(ctx, a, GGML_TRI_TYPE_LOWER_DIAG). This applies to a contiguous, square F32 tensor that is not modified in place
- Replace ggml_diag_mask_inf before a softmax with an additive mask passed to ggml_soft_max_ext (0 for visible positions, -INFINITY for masked ones)

The PR states that no model in the repository uses the removed ops. That applies to llama.cpp only; your own model implementations and old tutorial code are not covered. Older changes mentioned in the header's outdated comment, such as ggml_set_param no longer taking a context, ggml_graph_compute_with_ctx moving to the CPU backend header, and the optimization API moving to ggml-opt.h, can also trip up legacy code.

If you run distributed inference over RPC, take the most care. The tensor op enum is sent over the RPC wire, so removing ops shifts the numbers of the ops that follow. The RPC protocol major version was therefore bumped, and the handshake rejects peers unless the major version matches exactly; the patch version is not compared. Instead of misbehaving silently, the connection fails, which is the safe outcome, but mixed generations will not work. Rebuild the client and every rpc-server from the same generation and switch them over together.

Build and update steps

A minimal standalone ggml build follows the official README.

git clone https://github.com/ggml-org/ggml
cd ggml

mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

Most people use ggml through llama.cpp, so the actual update is pulling llama.cpp up to a newer master (or a matching release tag). The following is a typical flow; choose backend options for your environment, and treat llama.cpp's own documentation as authoritative for CUDA, Vulkan, and other build options.

cd llama.cpp
git pull

# Recreate the build directory so stale settings do not linger
rm -rf build
cmake -B build
cmake --build build --config Release -j 8

# Smoke test (adjust the model path)
./build/bin/llama-cli -m /path/to/model.gguf -p "Hello" -n 32

If you use RPC, repeat the same steps on every node, start the servers (rpc-server) first, and start the client last. A single node with an old build will be rejected at the handshake. This rejection behavior is based on the description of PR #31946.

# Rebuild with RPC enabled on every node
cmake -B build -DGGML_RPC=ON
cmake --build build --config Release -j 8

# Worker side
./build/bin/rpc-server -p 50052

# Client side (example)
./build/bin/llama-cli -m /path/to/model.gguf --rpc 192.168.0.10:50052 -p "Hello"

The option names above (-DGGML_RPC=ON, rpc-server, --rpc) are the ones llama.cpp has traditionally used. We did not see any note saying they were renamed in this release, but check --help on your build to be safe.

Migration checklist

- Identify whether you use llama.cpp directly or through another tool such as Ollama or LM Studio
- If you build llama.cpp yourself, follow the latest master and recreate the build directory
- In code that calls the ggml API directly, search for the removed names (GGML_OP_ADD1, ggml_mul_mat_set_prec, ggml_rope_custom, ggml_upscale_ext, ggml_flash_attn_ext_set_prec)
- Replace ggml_diag_mask_zero / ggml_diag_mask_inf with ggml_tri or an additive mask for ggml_soft_max_ext
- For RPC setups, build the client and all rpc-servers from the same generation and switch together
- For long-context (YaRN) models, compare output for the same prompt before and after, and confirm your backend supports the truncate setting
- Note your current build and a known-good tag before updating so you can roll back

Related articles

For a CPU-only application of llama.cpp, see running a 284B MoE on 64GB RAM without a GPU with llama-modchip. For choosing among local LLMs, see the April 2026 local LLM landscape. For an Ollama-based setup, see the Qwen 3.5-9B migration guide.

FAQ

Will ggml 1.0.0 break Ollama or LM Studio?

Unless you write code that calls ggml directly, your own code will not break. Which ggml revision each tool bundles depends on its own schedule and we could not confirm it from official sources, so check each tool's release notes.

I run distributed inference over RPC. Can I update only some nodes?

No. The op enum values changed, so the RPC major version was bumped, and the handshake rejects peers whose major version does not match. Build the client and all rpc-servers from the same generation and switch them together.

Does ggml 1.0.0 make inference faster?

Many backend optimizations are included, such as CUDA few-row MMQ, Metal MoE fusion, and Vulkan FlashAttention prefill. The official release notes give no speed numbers, so the gain depends on your environment and model and you need to measure it yourself.

How do I replace the removed ggml_diag_mask_zero?

Per PR #31946, replace ggml_diag_mask_zero(ctx, a, 0) with ggml_tri(ctx, a, GGML_TRI_TYPE_LOWER_DIAG) for a contiguous, square F32 tensor not modified in place. Express ggml_diag_mask_inf before a softmax as an additive mask to ggml_soft_max_ext.

What license is ggml under?

The README states MIT. The GitHub release page itself shows no license, so also check the LICENSE file in the repository before use.

Sources

- ggml v1.0.0 release notes: https://github.com/ggml-org/ggml/releases/tag/v1.0.0
- ggml README: https://github.com/ggml-org/ggml
- llama.cpp PR #29801 (ggml : prune): https://github.com/ggml-org/llama.cpp/pull/29801
- llama.cpp PR #31946 (remove DIAG_MASK): https://github.com/ggml-org/llama.cpp/pull/31946

株式会社オブライト

Oflight Inc. is a Tokyo-based IT company providing software development, digital signage, web development, network systems, and IoT/AR/VR technology services.

Feel free to contact us

Contact Us