Skip to main content
株式会社オブライト
株式会社オブライト

Microsoft Aion 1.0 Instruct 2.3B Requirements

Memory, Usage and Phi-4-mini Comparison (2026)

Published 2026/11/2015 min read

Microsoft Aion 1.0 Instruct (2.3B, MIT) needs about 4.6GB in BF16 and 1.1GB at 4-bit, plus ~1.1GB KV at 32K. Usage, Phi-4-mini comparison, Windows. Nov 2026.

Aion 1.0 Instruct (microsoft/Aion-1.0-Instruct-2.3B) is a 2.29B-parameter small model whose weights alone take about 4.6GB in BF16, about 2.3GB at 8-bit and about 1.1GB at 4-bit. Even with a 32K-token context, the KV cache comes to roughly 1.1GB (estimated), so a 4-bit build should fit on a PC with 4GB of RAM. It was built by the Applied Sciences Group in Microsoft's Windows & Devices organization and released on November 18, 2026 on Hugging Face as MIT-licensed open weights. It has gone GA as Edge's built-in model, replacing Phi-4-mini, and is planned to become Windows' built-in SLM as well. This article draws on the model card, config.json, the Microsoft Edge Blog and the Edge developer docs to work out memory requirements, show how to run it, and compare it with Phi-4-mini.

Information is current as of November 20, 2026 and comes from primary sources: the Hugging Face model card and config.json, the Microsoft Edge Blog post dated November 19, 2026, and the Prompt API documentation on Microsoft Learn. Memory figures computed from the parameter count are labeled as estimates, and items that officials do not state are marked as not documented.

Requirements at a glance

PrecisionWeight size calculationWeights onlyWith 8K context (approx.)With 32K context (approx.)Suitable hardware
BF16 (official release)2.29B x 2 bytes~4.6GB~5.0GB~5.7GBGPU with 8GB VRAM, PC or Mac with 16GB RAM
8-bit (INT8 / Q8)2.29B x 1 byte~2.3GB~2.7GB~3.4GBGPU with 4GB VRAM, PC or Mac with 8GB RAM
4-bit (INT4 / Q4)2.29B x 0.5 bytes~1.1GB~1.6GB~2.3GBPC with 4GB RAM, CPU-only possible

The formula is parameter count x bytes per parameter. The official count is 2,288,831,488 (about 2.29B); BF16 uses 2 bytes, giving about 4.58GB. The official file (model.safetensors) is shown on Hugging Face at about 4.59GB, which matches. The 8-bit and 4-bit values are theoretical; real quantized files run a few percent to 20% larger depending on the scheme (for example, 4-bit Q4_K_M averages roughly 4.8 bits per weight, about 1.3 to 1.4GB; all estimates). The two right-hand columns add the KV cache figures calculated below (about 0.4GB at 8K, about 1.1GB at 32K) and exclude runtime work buffers. In practice, leave a further 20 to 30 percent of headroom.

If you want to check quickly whether your GPU or Mac can run it, the free, no-signup VRAM calculator estimates required VRAM from the model, quantization and context length you choose. Aion 1.0 Instruct 2.3B is available in its model list.

What Aion 1.0 Instruct is and what was released

Aion 1.0 Instruct is a 2.3B-parameter instruction-tuned model built for on-device use on Windows. According to the model card, it was developed by the Applied Sciences Group at Microsoft Windows & Devices. Its training data extends the data used for Phi Silica, the previous-generation Windows built-in SLM: quality-filtered public documents plus synthetic data for math, coding, common-sense reasoning and general knowledge. Multilingual data is stated to be about 6% of the total.

It has three roles. First, MIT-licensed open weights on Hugging Face. Second, Microsoft Edge's built-in SLM: the Edge Blog (November 19, 2026) says Aion Instruct replaces Phi-4-mini as Edge's built-in model and is GA. Third, Windows' built-in SLM: the model card says a further optimized version, paired with a content moderation model, is planned to become the Windows OS SLM by the end of January 2027. The Edge Blog, by contrast, says only that it is coming to Windows later this year, so the two timelines are worded differently. No firm date is documented, so as of November 20, 2026 we stick to the model card's wording of a plan for the end of January 2027.

The release is the unquantized model; the model card says quantized versions will follow. It states that the Hub has eight quantized models usable in llama.cpp, Ollama, LM Studio and Jan. Of those, the one we could confirm as Microsoft's own is the ONNX WebGPU build (microsoft/Aion-1.0-Instruct-2.3B-ONNX-WebGPU); the GGUF and MLX builds we found were mostly third-party. We could not confirm whether an official GGUF exists as of this writing.

Architecture and specifications

ItemValue (model card / config.json)
Parameters~2.29B (2,288,831,488; tied input/output embeddings)
TypeCausal LM, instruction-tuned
Layers28
Hidden size2048
MLP intermediate size8192 (SiLU)
AttentionGQA (24 query heads, 8 KV heads, head dim 128)
Attention layoutHybrid: 21 sliding-window layers (window 2048) + 7 full-attention layers
Positional encodingRoPE (theta = 32000); YaRN (factor 16) on full-attention layers
Context length32,768 tokens
Vocabulary200,029
Precisionbfloat16
Knowledge cutoffDecember 2025
LanguagesEnglish, German, Spanish, Portuguese, Italian, French, Chinese, Japanese (8)
LicenseMIT

The vocabulary is notably large. 200,029 x hidden size 2048 gives roughly 410M parameters in the embedding layer alone (about 18% of the model), which stays that small only because input and output embeddings are tied; around a fifth of the weight size is vocabulary. The other notable point is that 21 of the 28 layers use sliding-window attention with a 2048 window, leaving only 7 full-attention layers, which keeps the KV cache small as shown in the next section. The model also supports tool calling: tools defined between <|tool|> markers are invoked by the model in <|tool_call|> format.

KV cache calculation: about 1.1GB even at 32K

Per token and per layer, the KV cache is K and V (2) x 8 KV heads x head dim 128 x 2 bytes (BF16) = 4,096 bytes (4KiB). The table below assumes the 7 full-attention layers store the whole context and the 21 sliding-window layers store only the 2048-token window (estimates; the official sources do not state a KV cache size).

Context length7 full-attention layers21 window layers (window 2048)KV total (BF16)
4,096~0.12GB~0.18GB~0.29GB
8,192~0.23GB~0.18GB~0.41GB
32,768~0.94GB~0.18GB~1.12GB

The full-attention part is 7 layers x 4,096 bytes x token count, about 0.94GB at 32,768 tokens. The window part is 21 layers x 4,096 bytes x 2,048 tokens, about 0.18GB, for a total of about 1.12GB. Some runtimes, however, allocate the full length even for window layers. In that case all 28 layers count as full attention and the cache grows to about 3.8GB at 32K and about 0.9GB at 8K. Check the actual figure with the runtime you use. Models of similar size with plain attention in every layer often need several GB at 32K, so this design helps when processing long documents locally; the model card itself lists on-device processing of long documents (up to 32K tokens) as an intended use.

Notes by environment

NVIDIA GPUs. Running BF16 in transformers needs about 4.6GB of weights plus about 1.1GB of KV at 32K plus work buffers, so a GPU with 8GB of VRAM is comfortable. With 8-bit or 4-bit builds, 4GB of VRAM should be enough by calculation. The model card also includes serving instructions for vLLM and SGLang (both with OpenAI-compatible APIs). Apple Silicon (Mac). Even with 8GB of unified memory, a 4-bit build leaves room after 1.1GB of weights and the context. MLX 4-bit and 8-bit builds are published by third parties. CPU only. A 4-bit GGUF in llama.cpp should run without a GPU, but tokens-per-second figures are not documented officially and depend on the machine.

Hardware requirements when built into Edge. These are separate from the model's own requirements. The Prompt API documentation on Microsoft Learn lists: Windows 10/11, macOS 13.3 or later, or x64 Linux with Vulkan GPU drivers and Edge 159 or later; at least 20GB free on the volume holding the Edge profile (the model is deleted if free space drops below 5GB); a GPU with 3GB of VRAM or more; and an unmetered network connection. You can check your device at edge://on-device-internals under Device performance class; Medium or higher should support the Prompt API. The documentation does not state whether these requirements were updated specifically for Aion Instruct.

Quick start with transformers

According to the model card, the model was exported with transformers 5.x and uses custom modeling code, so trust_remote_code=True is required. The minimal example below follows the model card's steps; it has not been run on real hardware.

pip install -U transformers accelerate torch
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = 'microsoft/Aion-1.0-Instruct-2.3B'
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map='auto',
    trust_remote_code=True,
)

messages = [
    {'role': 'user', 'content': 'Summarize the following text in three lines.'},
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors='pt', return_dict=True
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][inputs['input_ids'].shape[-1]:], skip_special_tokens=True))

The chat template uses the role tokens <|system|>, <|user|> and <|assistant|>, with each turn closed by <|end|> (which is also the EOS token). A leading system message is appended to the default system preamble. The official steps for serving it are below.

# vLLM (OpenAI-compatible API)
vllm serve microsoft/Aion-1.0-Instruct-2.3B --trust-remote-code

# Docker Model Runner
docker model run hf.co/microsoft/Aion-1.0-Instruct-2.3B

The vLLM options are shown in a common form, and whether --trust-remote-code is needed may vary by version. The model card also gives steps for SGLang (sglang.launch_server). For llama.cpp or Ollama, the model card says the quantized models work with llama.cpp, Ollama, LM Studio and Jan, but most of the quantized builds we found are in third-party repositories, so follow each repository's instructions for exact commands. We could not confirm whether it is listed in the official Ollama library as of this writing.

Using it in Edge: the Prompt API and Decisions API

In Edge, page JavaScript can call the built-in model through the LanguageModel object. According to the Edge Blog, the Prompt API was previously a developer preview in the Canary and Dev channels only, and has now moved to an origin trial (registration required), so it can be tried on live sites. The model is downloaded only when a site or feature first needs it, and only on devices that can run it. Turning off on-device AI in Edge settings removes the model and stops future downloads.

if (typeof LanguageModel !== 'undefined') {
  const availability = await LanguageModel.availability();
  if (availability !== 'unavailable') {
    const session = await LanguageModel.create({
      initialPrompts: [{ role: 'system', content: 'You are a helpful assistant.' }],
    });
    const result = await session.prompt('Summarize this page in one sentence.');
    console.log(result);
    session.destroy();
  }
}

The Microsoft Learn documentation says you can constrain output by passing a JSON schema or regular expression as responseConstraint. Also, from Edge 158, text generated by the Prompt API carries an invisible watermark marking it as AI-generated, which is not applied to responses that use responseConstraint. Code that processes generated strings has to account for it. The Edge Blog also introduces an experimental Decisions API (Canary and Dev only) powered by Aion Instruct: for yes/no, multiple-choice and scoring questions about text, it returns probabilities and a confidence value in a single pass without generating text. A hybrid setup that can fall back to a configured cloud provider is described as well.

Benchmarks and comparison with Phi-4-mini, Gemma and Qwen

These are the official figures on the Aion 1.0 Instruct model card. Benchmarks were run with EleutherAI's lm-evaluation-harness, or each benchmark's official implementation where no harness task existed, and GPT-4o served as the judge for LLM-judged evaluations. Knowledge: MMLU 55.15, MMLU-Pro 33.35, MMMLU (multilingual MMLU) 45.51, GPQA 28.18. Math: GSM8K 74.91 (loose matching), GSMPlus 49.56, MATH-500 48.00. Instruction following: IFEval 83.90, IFBench 32.00, Multi-IF 64.70. Translation: chrF++ 43.18.

ItemAion 1.0 InstructPhi-4-mini-instruct
Parameters~2.29B3.8B (HF sidebar shows 4B)
Context length32K128K
LicenseMITMIT
Languages8 (incl. Japanese)23
MMLU55.1567.3 (5-shot)
MMLU-Pro33.3552.8 (0-shot, CoT)
Multilingual MMLU45.51 (MMMLU)49.3 (5-shot)
GPQA28.1825.2 (0-shot, CoT)
GSM8K74.91 (loose matching)88.6 (8-shot, CoT)
Math48.00 (MATH-500)64.0 (MATH, 0-shot, CoT)
ReleasedNovember 2026February 2025

The Phi-4-mini figures come from that model's own card, and evaluation conditions (shot counts, judging, MATH versus MATH-500) are not aligned, so the numbers cannot be compared directly. Even so, reading the official figures side by side, Aion is generally below Phi-4-mini on knowledge and math and ahead only on GPQA. The Edge Blog's wording is that it is smaller, faster and available on many more devices than Phi, with comparable quality for common web tasks; it does not claim higher overall capability. Concrete speed or memory advantages are not documented in numbers. The sensible reading is that Aion targets running everyday tasks such as summarization, rewriting and structured output on more devices, rather than peak accuracy.

We could not confirm official same-condition benchmarks against Gemma or Qwen small models, so we make no claims about which is better. For memory requirements of similar-size models, see our guides: Gemma 4 requirements, the edge-oriented Gemma 4 E4B guide, MiniCPM5-2B requirements and LFM3 requirements.

Using it in Japanese

Japanese is among the supported languages. However, the model card states that output quality varies by language, and with multilingual data at only about 6% of training, you should verify Japanese accuracy on your own data before adopting it. We could not find official benchmarks specific to Japanese. The large 200,029-token vocabulary may make Japanese token-efficient, but there is no official statement, so measure token counts on your own text.

Commercial use and licensing

The MIT license permits commercial use, modification and redistribution, with none of the revenue-based conditions seen in licenses like LFM3's. MIT does require that the copyright notice and license text be included, so include the repository's LICENSE file when redistributing or embedding the model. The model card also lists out-of-scope uses: generating harmful, violent, explicit or offensive content; misleading content such as fake news; and automated decisions in critical contexts such as medical diagnosis or legal judgments. Developers are asked to evaluate accuracy, safety and fairness for their use case and comply with applicable laws.

As limitations, the model card admits that at 2.3B parameters it may miss details, make factual errors, struggle with complex multi-step reasoning, have limited world knowledge and be susceptible to adversarial attacks. Windows plans to pair it with a content moderation model, but that safety layer does not ship with the standalone open weights. If you embed it in your own service, set up input and output filters and human review separately. Also note that trust_remote_code=True executes Python code from the model repository. In production, pin a revision (commit hash) and review the code before using it.

What to do when you run out of memory

- Shorten the context length. You do not need all 32K; limiting to 8K cuts the KV cache to about 0.4GB (estimated)
- Lower the quantization. Moving from BF16 to 8-bit or 4-bit halves or quarters the weights
- If your runtime can quantize the KV cache to 8-bit, you can halve the KV size
- Use CPU offload with device_map='auto', at a large cost in speed
- A runtime that allocates full length for window layers makes KV about 3 times larger; switch runtimes or shorten the context
- Check your transformers version. The model was exported with 5.x

Where small models fit

Aion suits summarization, rewriting, classification and simple structured output that should stay on the device, plus light assistive features inside the browser. Advanced reasoning, questions needing broad knowledge and very long contexts beyond 128K are better handled by larger models or cloud APIs. For how to split work between local and cloud, see hybrid AI cost reduction with cloud and local LLMs; for adopting SLMs in general, see the SLM business guide; and for choosing a runtime, see the local LLM inference engine comparison.

FAQ

How much memory does Aion 1.0 Instruct 2.3B need?

Calculated from 2.29B parameters, weights are about 4.6GB in BF16, 2.3GB at 8-bit and 1.1GB at 4-bit (estimates). The KV cache is about 1.1GB at 32K tokens (estimated, assuming window layers keep only their window). Leaving 20 to 30 percent headroom, plan for about 4GB of RAM at 4-bit and 8GB of VRAM for BF16.

Can Aion 1.0 Instruct be used commercially?

It is released under the MIT license, which allows commercial use, modification and redistribution, provided the copyright notice and license text are included. The model card lists generating harmful content, misinformation and automating medical or legal decisions as out-of-scope uses.

Is Aion 1.0 Instruct better than Phi-4-mini?

Reading the official figures side by side, it scores below Phi-4-mini's model card on MMLU, MMLU-Pro and GSM8K and above on GPQA (evaluation conditions are not aligned). The Edge Blog describes it as smaller, faster and available on more devices than Phi with comparable quality for common web tasks, and does not claim higher overall accuracy.

Does it support Japanese?

Japanese is among its 8 supported languages. The model card says quality varies by language, and we could not confirm any Japanese-specific official benchmarks, so test on your own data before adopting it.

When will it be built into Windows?

The model card says an optimized version is planned to become the Windows OS SLM by the end of January 2027. The Edge Blog says it is coming to Windows later this year, so the wording differs. No firm date is documented.

Can I use it with Ollama or llama.cpp?

The model card says the quantized models work with llama.cpp, Ollama, LM Studio and Jan. However, the GGUF and MLX builds we found were mostly in third-party repositories, and we could not confirm an official GGUF or an official Ollama library entry as of this writing.

株式会社オブライト

Oflight Inc. is a Tokyo-based IT company providing software development, digital signage, web development, network systems, and IoT/AR/VR technology services.

Feel free to contact us

Contact Us