On-Device LLMs for React Native

Run powerful language models directly on mobile devices with Google's MediaPipe Inference Task API. Fast, private, and works offline.

import { useLLM } from 'expo-llm-mediapipe';
                  
function LlmInference() {
  const { generateResponse, isLoaded } = useLLM({
    modelUrl: 'https://huggingface.co/google/gemma-1.1-2b...',
    modelName: 'gemma-1.1-2b-it-cpu-int4.bin',
  });

  if (isLoaded) {
    const response = await generateResponse(
      'Tell me about React Native'
    );
    console.log(response);
  }
}

Simple API

Key Features

On-Device Intelligence

Run LLM inference directly on the device without relying on external servers.

Cross-Platform Support

Works seamlessly on both Android and iOS devices.

Streaming Generation

Generate token-by-token responses for responsive UIs.

Download Management

Download, cancel, or delete models on-device with ease.

React Hooks

Simplified APIs for declarative usage within your React Native application.

Manual APIs

Advanced APIs for developers who need more control over the inference process.

Getting Started

expo install @expo/mediapipe-llm

bash

Basic Usage

import { useState } from 'react';
import { View, Text, Button } from 'react-native';
import { useLLM } from 'expo-llm-mediapipe';

export default function App() {
  const [response, setResponse] = useState(null);
  const [isGenerating, setIsGenerating] = useState(false);
  const { 
    generateResponse, 
    isLoaded,
  } = useLLM({
    storageType: 'asset',
    modelName: 'gemma-1.1-2b-it-cpu-int4.bin',
    maxTokens: 1024,
    temperature: 0.5,
    topK: 1,
  });

  return !isLoaded && (
    <Text>Loading...</Text>
  );

  return (
    <View style={{ flex: 1, padding: 20 }}>
      <Button 
        onPress={() => {
          setIsGenerating(true);
          const res = generateResponse("Tell me about React Native");
          setResponse(res);
          setIsGenerating(false);
        }}
        disabled={isGenerating}
        title={isGenerating ? "Generating..." : "Generate Text"}
      />
      
      {result && <Text>{result}</Text>}
      {error && <Text style={{ color: 'red' }}>{error}</Text>}
    </View>
  );
}

jsx

What is MediaPipe LLM Inference Task?

The MediaPipe LLM Inference API lets you run large language models (LLMs) completely on-device, which you can use to perform a wide range of tasks:

Text generation based on prompts
Retrieving information in natural language form
Summarizing documents and other content
Running models like Gemma, Phi-2, and others efficiently on mobile hardware
Processing inputs offline with complete privacy

This package provides a React Native wrapper around MediaPipe's LLM inference capabilities, making it easy to integrate powerful on-device language models into your mobile applications with no server dependencies.

Documentation

API Reference

Complete documentation of all available APIs and options

Examples

Code examples and demos to help you get started

Guides

Step-by-step guides for common use cases

Supported Models

List of LLM models compatible with this package