Index-Translate Requirements: VRAM & GPU for 2B/9B/35B

Index-Translate (bilibili / IndexTeam) is a family of open-weight translation models built on Qwen3.5 and covering 150 languages. As a rough guide it needs about 2–5GB for the 2B model, 6–19GB for the 9B, and 21–72GB for the 35B-A3B, depending on quantization. The 2B runs on an 8GB GPU, a Mac, or even a CPU, the 9B is practical on a 16GB GPU, and the 35B-A3B fits a 24–32GB GPU or a Mac with 32GB+ memory at 4-bit. It is released under Apache 2.0, so commercial use is allowed (as of October 2026).

This article covers the requirements table, memory estimates per quantization (GGUF/FP8/NVFP4), how to run it on NVIDIA GPUs, Apple Silicon and CPUs, the benchmarks, a vLLM how-to, and a comparison with other translation options. For the basics of running a model locally, see the Qwen3.5 9B local setup guide. Note that the memory figures are our own estimates computed from weight sizes, not official measurements.

The table shows estimates for weights only. In practice the KV cache and runtime overhead add a few GB, and more as the context grows. Adding roughly 2–4GB to the numbers above is a sensible margin.

To check quickly whether a given GPU or Mac can run a model, the VRAM Calculator (free, no sign-up) estimates the VRAM you need once you pick a model, quantization, and context length. Index-Translate 2B, 9B and 35B-A3B are in its model list.

Index-Translate is a translation-focused open-weight model family released by the Index LLM team (IndexTeam) at video platform bilibili. It is built on Qwen3.5 and comes in three sizes: 2B, 9B and 35B-A3B. The code is at bilibili/Index-Translate on GitHub, and the weights are on Hugging Face (IndexTeam/Index-Translate-35B-A3B-preview, IndexTeam/Index-Translate-9B, IndexTeam/Index-Translate-2B) and ModelScope.

The release sequence was as follows. On September 30, 2026 the text-translation weights (2B, 9B and the 35B-A3B preview) went out. On October 3 official quantized builds were added: GGUF for llama.cpp, plus FP8 W8A8 and NVFP4 W4A4 for vLLM. NVFP4 targets Blackwell GPUs. On October 4 a free OpenAI-compatible public API (https://index-translate.bilibili.com/v1, serving the 35B-A3B) and the benchmark datasets were released.

It supports text translation across 150 languages, Japanese included. Its other distinguishing feature is instruction-following translation: you can specify hard constraints (glossary and terminology enforcement, format preservation) and soft constraints (tone, domain disambiguation, length) in the prompt. The family also includes Index-Echo (speech translation, S2TT/S2ST, between Chinese and English/Japanese/Spanish), Index-Homura and Index-NativeLong (full-document translation), but this article focuses on the three text models.

At 4–5GB in BF16 and around 2GB at 4-bit, it runs on an 8GB GPU, an 8GB Mac, or even a CPU alone. The trade-off is quality: its WMT26 judge score is 60.26, nearly 15 points below the 9B's 75.35. Think of it as a model for on-device use, embedded scenarios, or fast first-pass drafts.

The 9B needs about 18–19GB in BF16, about 10GB in FP8, and about 6GB at Q4_K_M. An RTX 4060 Ti 16GB or RTX 5060 Ti 16GB handles Q4–Q8 comfortably. An RTX 4090/5090 (24/32GB) can hold BF16, but long contexts squeeze the KV cache, so FP8 is safer. On a Mac, 16GB with Q4 is the realistic floor. As the benchmarks below show, it is nearly level with the 35B-A3B on FLORES, which makes it very cost-effective.

It is an MoE with about 35B total parameters (the Hugging Face card lists about 36B) and only about 3B active at inference. BF16 weights are about 70–72GB, requiring an 80GB H100/A100 or a Mac with 96–128GB. FP8 is about 36–38GB and runs on an RTX 6000 Ada (48GB) or two 24GB cards. The 4-bit GGUF is about 21–23GB and fits an RTX 4090/5090 or a Mac with 32GB+.

The advantage of MoE is that you need the memory for the full model, but each token costs only about 3B parameters of compute. That helps keep tokens per second up even when part of the model is offloaded to CPU memory, compared with a dense model of the same size. As the name says, this is a preview release, so weights and performance may change in the final version. Maximum position embeddings are 262,144, but the recommended serve setting is --max-model-len 32768, and evaluation used 16,384.

The official quantized builds were added on October 3, so there is no need to make GGUF files yourself. As a rule of thumb: GGUF for a single GPU with limited VRAM, FP8 for a vLLM server, and NVFP4 for Blackwell hardware (RTX 50 series, B200 and so on). Because a single wrong word stands out in translation, check on your own languages and style whether 4-bit is good enough. For differences between llama.cpp, vLLM and others, see the local LLM inference engine comparison.

Macs use unified memory, so the memory available to the GPU effectively sets the model ceiling. Because the OS and apps take a share, plan on roughly 70–80% of installed memory. A 16GB Mac runs the 9B at Q4 (about 6GB) comfortably, and the 2B runs on any Mac. With 32GB or more, the 35B-A3B at Q4 (about 21–23GB) is feasible, and 96–128GB opens up BF16 (about 70–72GB). Since the MoE computes only 3B parameters per token, speed drops less on a Mac than you might expect. Besides llama.cpp, GGUF-capable tools such as LM Studio and Ollama work too (see Ollama vs LM Studio).

These are estimates covering weights plus KV cache, and you should leave headroom if you plan to use the full 32,768-token context. Translation is often done paragraph by paragraph, so many real workloads never need a long context.

The 4-bit 2B runs under llama.cpp on an ordinary laptop with no GPU. The 9B is about 6GB, so it works on a CPU machine with 16GB of RAM, though slowly. The 35B-A3B is about 21–23GB at 4-bit and needs 32GB+ of RAM, but because it is an MoE with 3B active, it tends to reach more realistic CPU speeds than a dense model of the same size. If the model does not fit in VRAM, putting some layers on the GPU and the rest on the CPU is also effective.

- Existing PC or Mac: the 2B at 4-bit. Works with 8GB of memory; good for drafts and light translation
- A PC with a 16GB GPU, or a 16GB Mac: the 9B at Q4–Q8. The best balance
- RTX 4090/5090, or a Mac with 32GB+: the 35B-A3B as 4-bit GGUF. Quality-focused local operation
- Business server (48GB-class GPU): the 35B-A3B in FP8 served with vLLM, shared by several users
- 80GB GPU (H100/A100): the 35B-A3B in BF16, for accuracy-first evaluation

The numbers below come from the Hugging Face model card and the repository, and are vendor-reported by bilibili. As of this writing we have not confirmed independent reproductions.

Among the comparisons the repository reports, GPT-5.6-Sol scores 89.10 on WMT26, well above the 35B-A3B's 76.76. In overall translation quality, the top closed model still has a clear lead. On MEME (internet slang and cultural expressions), however, the 35B-A3B scores 0.7405 against GPT-5.6-Sol's 0.7194. The repo also reports Hy-MT2-7B at 0.8747 on FLORES, DeepSeek-V4.1-Flash at 0.6068 on instTrans Quality, and Gemini 3.5 Flash at 0.6374 on IFscore, so Index-Translate's constraint-following is ahead of those comparison points.

Two takeaways. First, the 9B is nearly level with the 35B-A3B on FLORES and MEME and only about 1.4 points behind on WMT26, which makes it the better value. Second, the 2B falls well behind, so it is hard to recommend where quality matters.

The recommended inference settings are greedy decoding (temperature 0), max_tokens of 1,024 by default, and thinking disabled. Here is an example vLLM launch.

Once running, you can call it from any OpenAI-compatible client. To turn thinking off, pass enable_thinking=False in chat_template_kwargs via extra_body.

You can also use the CLI that ships with the repository.

The official prompt template is written in Chinese. Standard translation uses the following form, with language names in {source} and {target} and the source text in {text}.

To add constraints such as glossaries or formatting, use the 【源文】 / 【约束要求】 format and list hard constraints as numbered 【硬性要求】 items. The following is a skeleton to show the layout; replace the constraint wording to suit your use.

Since October 4, a free OpenAI-compatible API has been available at https://index-translate.bilibili.com/v1 (serving the 35B-A3B). You can try it just by changing the client's base_url. However, details such as rate limits and terms of use are not documented in detail at this time. How text sent to a public endpoint is handled is not guaranteed, so do not send confidential documents or personal data; for those, run the model locally.

How to choose: if you need the highest quality and confidentiality is not an issue, a top-tier API such as GPT-5.6-Sol has the edge (89.10 on WMT26). Index-Translate fits when drafts cannot leave your network, when API costs balloon at volume, or when you need terminology enforced strictly. DeepL and Google are strong on steady production quality and easy-to-manage features such as glossaries.

- Out of memory (OOM): lower --max-model-len (32768 to 16384), switch to an FP8 or GGUF build, or use a model of 9B or smaller. Adjusting gpu-memory-utilization in vLLM can also help
- Chinese leaks into the output: the template is in Chinese, so always fill in {target} and name the target language explicitly, such as English or Japanese. Off-target output (text in an unintended language) is reported at 2.4% for low-resource directions, so adding a language-detection check afterward is reassuring
- Long reasoning text or slow responses: always pass enable_thinking=False, and use temperature 0 (greedy decoding)
- Multiple constraints are not respected: the model card itself notes that combined constraints and low-resource directions are harder. Reduce the number of constraints, or add a script that verifies terminology after translation
- Quality is lower than expected: keep in mind that this is a preview and that the 2B drops off sharply, and try 9B or larger

Index-Translate is an Apache 2.0, 150-language translation model family, and the 2B to 35B-A3B range lets you match the model to your hardware. The 9B runs on a 16GB GPU with strong quality, and the 35B-A3B runs on a 24–32GB GPU at 4-bit. It does not reach GPT-5.6-Sol's quality ceiling, but its value lies in constraint-aware translation of terminology and format, and in handling confidential documents locally. A good first step is to test a quantized 9B on your own text.