Mobile Development2026-07-09

React Native ExecuTorch's useLLM Hook Deep Dive — Run Qwen / Llama 3.2 / Hammer 2.1 / Phi 4 Mini / SmolLM 2 / LFM2.5 / Gemma 4 On-Device in React Native, With Tool Calling, Vision / Audio, and Structured Output From Software Mansion, Two Modes (Managed / Functional), and Zod-Backed Schema Validation

Software Mansion's React Native ExecuTorch shipped a useLLM hook that gives React Native apps native on-device LLM integration. Supported models: quantized Qwen (2.5 / 3 / 3.5), Llama 3.2, Hammer 2.1, Phi 4 Mini, SmolLM 2, LFM2.5 (vision), and Gemma 4 (vision + audio). Two operating modes: Functional/Stateless (developers manage conversation history via generate() + response; tool calling and chat config don't apply) and Managed/Stateful (sendMessage() maintains conversation state, parses tool calls, and runs callbacks automatically). Key features: token batching (groups tokens before re-render), tool calling (model invokes external functions via tool schemas with automatic parsing and callbacks), vision-language / audio multimodal inputs, generation control (temperature / top-p / repetition penalty / mid-stream interruption), and JSONSchema- or Zod-backed structured output. Use cases: on-device chatbots without server dependencies, privacy-first conversation UIs, in-app function calling (calendar events, flashlight, etc.), multimodal features (image analysis, audio transcription). Positioning: implementation-level evidence that cutting-edge open-weight LLMs like Gemma 4's encoder-free 12B and Qwen 3.6-35B natively fit into iOS / Android apps — a major deliverable from the React Native ecosystem.

React Native ExecuTorch On-Device LLM Software Mansion Qwen Llama Gemma 4 Mobile AI

TL;DR — React Native ExecuTorch × useLLM

Software Mansion's React Native ExecuTorch wraps Meta's ExecuTorch (PyTorch's on-device runtime) for React Native. The useLLM hook lets you run on-device LLMs from tens of lines of React Native code.

Four takeaways:

1. Rich model coverage — Qwen 3.5, Llama 3.2, Phi 4 Mini, SmolLM 2, LFM2.5, Gemma 4 (vision + audio)
2. Two modes — Functional/Stateless (low-level control) and Managed/Stateful (automatic management)
3. Tool calling + multimodality + structured output as first-class features
4. Fully on-device — no server, privacy-preserving, offline-capable

Supported Models

Model	Size band	Notes
Qwen 2.5 / 3 / 3.5	0.5B–7B	Alibaba general-purpose
Llama 3.2	1B / 3B	Meta lightweight
Hammer 2.1	—	Tool-calling focused
Phi 4 Mini	3.8B	Microsoft efficiency
SmolLM 2	135M–1.7B	Ultra-small (edge-friendly)
LFM2.5	—	Vision-enabled, Liquid AI family
Gemma 4	2.3B / 12B / 31B	Vision + audio, encoder-free 12B

All ship as quantized (GGUF-equivalent + ExecuTorch format) builds sized for real-device usability on iOS / Android.

Two Operating Modes

1. Functional / Stateless

Developer manages conversation history, using generate() and response.

tsx

const llm = useLLM({ modelSource, tokenizerSource, tokenizerConfigSource });

await llm.generate([
  { role: 'system', content: 'You are a helpful assistant.' },
  { role: 'user', content: 'Hello!' },
]);
// stream via llm.response

When to use: tool calling and chat config don't apply; useful for full control or integrating with existing chat state.

2. Managed / Stateful

Hook auto-manages conversation state, parses tool calls, executes callbacks. Just call sendMessage().

tsx

const llm = useLLM({ modelSource, tokenizerSource, tokenizerConfigSource });

llm.configure({
  chatConfig: { systemPrompt: 'You are a helpful assistant.' },
  toolsConfig: { tools: [openFlashlight, createCalendarEvent] },
  generationConfig: { temperature: 0.7 },
});

await llm.sendMessage('Turn on the flashlight.');
// The hook detects the tool call, runs openFlashlight, feeds the result back to the LLM for the final reply

When to use: standard chat UIs — history and function calling are handled for you.

Feature Deep Dive

Token Batching — Lower Re-Render Cost

LLM streams span hundreds to thousands of tokens; a re-render per token murders frame rate. useLLM groups tokens into batches before triggering re-renders, with batch size configurable via generationConfig. Determines mobile-UX feasibility at the implementation level.

Tool Calling — In-App Function Invocation

The model invokes external functions per your tool schema; useLLM parses and dispatches automatically:

tsx

const openFlashlight = {
  name: 'openFlashlight',
  description: 'Turn on the device flashlight.',
  parameters: { /* JSON Schema */ },
  execute: async () => { /* Native module call */ },
};

Calendar events, notifications, location, native module calls — the full range of a React Native app becomes controllable from the LLM.

Vision / Audio Multimodality

Vision-capable models (LFM2.5, Gemma 4) accept images via the capabilities array + media parameters; audio-capable models (Gemma 4) accept audio.

tsx

await llm.sendMessage('Describe this photo.', {
  images: [{ uri: 'file:///path/to/photo.jpg' }],
});

Use cases: summarizing a document photographed with the camera, receipt OCR, voice-memo transcription, image descriptions for accessibility.

Generation Control

Generation parameters:
- temperature — randomness
- top-p — nucleus sampling
- repetition penalty — anti-loop
- Mid-stream interruption — user hits "stop" mid-response

Being able to cancel a long response matters for mobile UX.

Structured Output — JSONSchema / Zod

Force a predictable JSON response:

tsx

import { z } from 'zod';

const schema = z.object({
  intent: z.enum(['play_music', 'open_map', 'send_message']),
  target: z.string(),
  urgency: z.number().min(1).max(5),
});

await llm.sendMessage('Play some jazz please.', {
  responseSchema: schema,
});

In-app intent parsing, autofilled forms, assistant features all get type-safe LLM responses. Both JSONSchema and Zod are supported.

Practical Use Cases

(1) On-device chatbots: no server, works offline, zero network latency.

(2) Privacy-preserving conversation UIs: for medical, legal, and finance apps — user data never leaves the device.

(3) Natural-language app control: "dentist at 3 PM tomorrow" auto-creates a calendar event; "turn on the flashlight" runs it.

(4) Action extraction from image / audio: a photo of a receipt becomes an expense entry; a voice memo becomes tasks.

(5) Multilingual assistants: real-time English / Japanese / Chinese switching, translation, summarization.

Performance Reference Points

iPhone 15 Pro (A17 Pro) hits tens of tokens/sec on Qwen 2.5-1.5B Q4; Google Pixel 9 Pro (Tensor G4) runs Llama 3.2-3B Q4 at 15–25 tokens/sec. On iPhone 17 Pro / Pixel 10 Pro / Snapdragon 8 Gen 5 class devices, Gemma 4-12B Q4 becomes practical. iOS Core ML backend, Android NNAPI backend, and CPU fallback are auto-selected.

Positioning — the Gateway to On-Device LLMs

The 2026 mobile-AI ecosystem:

1. iOS: Apple Intelligence — Apple's own on-device model; developer access via the Foundation Models API
2. Android: Gemini Nano — Google's own; developer access via AICore
3. React Native ExecuTorch + useLLM — cross-platform, free model choice: Qwen / Llama / Gemma 4 and other open weights

Distinctive advantage: Apple Intelligence and Gemini Nano lock you into their model; React Native ExecuTorch gives you both model choice and cross-platform code. A clean way to run Gemma 4 12B's encoder-free vision + audio natively on mobile.

Caveats and Warnings

(1) Model binary size vs app size: 3B Q4 is ~2 GB, 12B Q4 is ~7 GB. App-store limits (iOS 4 GB, Android 200 MB compressed) usually force first-launch download design.

(2) Battery: LLM inference maxes out CPU / GPU / NPU; continuous use drains batteries fast. Warn users, watch thermals.

(3) Memory footprint: with only 6–8 GB of RAM on iPhone, 3–5 GB goes to the LLM; be careful about coexisting with other memory-hungry pieces.

(4) Device fragmentation: low-spec devices may not hit usable speeds — pick model sizes per device.

(5) React Native ExecuTorch is still evolving — the API isn't stable yet, expect breaking changes.

Bottom Line

React Native ExecuTorch's useLLM hook is the implementation-level answer for embedding Gemma 4 / Qwen / Llama 3.2-class open-weight LLMs natively into iOS / Android apps. Tool calling, vision / audio, structured output, token batching, and mid-stream interruption — the things that decide mobile AI UX — all as first-class features. A clean path to server-free, privacy-preserving AI experiences.

Related services from us — mobile app development, AI consulting, and software development. For help building React Native + on-device LLM apps or designing Gemma 4 / Qwen-based mobile assistants, get in touch.

References

- React Native ExecuTorch — useLLM docs
- React Native ExecuTorch home
- PyTorch ExecuTorch
- Software Mansion
- Related columns: Gemma 4 Technical Report, Qwen 3.6-35B Uncensored, Liquid AI LFM2.5, Local LLM June 2026 Update

Feel free to contact us