LiteRT.js: Run ML Models in the Browser Without a Server

The Big Idea

On July 9, 2026, Google's Ping Yu, Marko Ristic, Matthew Soulanille, and Chintan Parikh announced LiteRT.js - a WebAssembly binding of Google's native LiteRT C++ inference runtime that brings on-device ML execution directly to the browser. Until now, running ML models in the browser meant TensorFlow.js, a JavaScript-based runtime with lower performance, no shared code with native mobile deployments, and limited hardware acceleration. LiteRT.js replaces the JavaScript kernel layer with the same C++ runtime that powers ML inference on Android, iOS, and desktop, exposed through a single @litertjs/core npm package. The result is up to 3x faster CPU inference compared to existing web runtimes, and 5-60x speedups when hardware acceleration is routed through WebGPU or the experimental WebNN API. This shifts the economics of deploying ML features: semantic search, object detection, image upscaling, and depth estimation all become client-side operations with zero server cost, zero API latency, and no data leaving the browser.

Before vs After

How It Works

The Runtime Stack

LiteRT.js is a thin WebAssembly layer over Google's existing LiteRT C++ runtime. The C++ runtime itself is not new - it has powered on-device inference on Android and iOS for years under the TensorFlow Lite name. What is new is the compilation target: Emscripten compiles the C++ runtime to WebAssembly, which the browser executes at near-native speed with access to web-specific hardware APIs. The CPU backend uses XNNPACK, an optimized micro-kernel library with multi-thread and SIMD support, running via the standard Web Workers and SharedArrayBuffer threading model available in modern browsers. The GPU backend routes through ML Drift, a WebGPU-based acceleration layer that dispatches compute shaders to the GPU directly from the browser. The NPU backend uses the WebNN API, which is currently experimental in Chrome and Edge only but provides a direct path to dedicated neural processing hardware when available.

Model Conversion Pipeline

The model format is .tflite - the same binary format used for on-device mobile deployments. PyTorch models convert to .tflite in a single step using LiteRT Torch, Google's purpose-built conversion library. Quantization is handled by AI Edge Quantizer, which supports layer-level configuration - you can apply int8 or float16 quantization selectively across layers rather than applying a uniform quantization strategy to the entire model. Pre-converted models are available on Kaggle and the LiteRT Hugging Face community, which means a web automation developer can pull a ready-to-use .tflite model without touching the training or conversion pipeline at all. The same .tflite file runs across Android, iOS, desktop, and browser, so a model validated on-device for a mobile application can be deployed to the browser without re-conversion.

What Runs Client-Side Now

The Google announcement demonstrated five categories of ML tasks running entirely in the browser. Semantic search uses EmbeddingGemma to produce dense vector embeddings from text queries, enabling search-over-documents without a vector database backend. Real-time object detection uses Ultralytics YOLO26 for frame-by-frame detection at video rates. Monocular depth estimation uses Depth-Anything-V2 with the WebGPU backend to produce per-pixel depth maps from single images, which can then be converted into interactive 3D point clouds in the browser. 4x image upscaling uses Real-ESRGAN to upscale 128x128 patches to 512x512 via patch-based inference. LLM inference is handled by a separate companion library, LiteRT-LM.js, which is not included in the core package but uses the same runtime infrastructure.

Hardware Acceleration in Practice

The 5-60x GPU vs CPU figure is the widest range in the announcement, and it reflects the variance in real-world GPU capability across consumer devices rather than a controlled experiment. On a 2024 Apple MacBook Pro M4, which is the benchmark hardware Google used, the WebGPU backend via ML Drift delivers performance near the top of that range for vision tasks. On integrated Intel or AMD graphics, the speedup is considerably lower. The WebNN backend, which targets dedicated neural processing hardware such as Apple Neural Engine or Qualcomm NPUs, is currently experimental in Chrome and Edge, meaning it is behind a feature flag and not production-ready for deployment. The CPU path via XNNPACK with multi-thread SIMD is the most reliable baseline: it is fully supported across all modern browsers that implement SharedArrayBuffer and Web Workers, which requires HTTPS and specific CORS headers (Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp).

Key Findings

• 3x CPU speedup over TensorFlow.js is the conservative baseline. The performance gap comes from replacing JavaScript kernels with compiled C++ via WASM, plus XNNPACK's SIMD-optimized micro-kernels. This holds across device classes where WebGPU is unavailable or unreliable.
• The .tflite format unifies the deployment target. One model file serves Android, iOS, desktop, and browser without re-conversion. This eliminates the platform-specific model management overhead that currently fragments on-device ML deployments.
• WebNN is the ceiling, not the floor. The NPU backend delivers the highest performance and lowest power draw, but it is experimental and limited to Chrome and Edge. Shipping on WebNN today means accepting a feature flag dependency; the safe production path is WebGPU with XNNPACK fallback.
• Quantization is layer-level, not model-level. AI Edge Quantizer allows mixed-precision quantization at the layer granularity. For automation practitioners, this means you can preserve float32 precision in accuracy-critical layers while quantizing compute-heavy backbone layers, without retraining.
• Benchmark numbers are from high-end hardware. All published performance figures come from a 2024 Apple M4. Real-world variance on mid-tier consumer hardware will be substantial, particularly on the GPU path. XNNPACK CPU performance is more consistent across device classes.

Why This Matters for AI and Automation Practitioners

The deployment model for ML features in web automation has been: train model, expose via API, call from browser. LiteRT.js makes a third option viable for a meaningful class of problems: run the model in the browser and skip the API entirely. The cases where this changes the economics most clearly are high-frequency, low-latency tasks - real-time form field extraction from uploaded documents, client-side image preprocessing before upload, semantic deduplication of search queries before they hit a backend, and on-device classification in environments where data privacy rules prohibit sending content to external servers. For practitioners building automation tools on top of web interfaces, the ability to embed a vision or embedding model directly into a browser extension or web app without provisioning inference infrastructure represents a genuine shift in what is buildable without backend complexity. The constraint is the model size that fits within browser memory limits and delivers acceptable latency on consumer hardware - the current sweet spot is models in the tens of MB range, well below the multi-GB frontier models but covering a large fraction of practical classification, detection, and embedding workloads.

My Take

The most significant thing about LiteRT.js is not the performance number - it is the unified model format. The fragmentation between TensorFlow.js models, ONNX Runtime Web models, and native on-device models has been a real friction point for teams building ML features across platforms. A single .tflite file that runs identically on Android, iOS, and browser removes that friction directly. The 3x speedup over TensorFlow.js is meaningful for CPU-bound workloads but becomes less relevant for GPU-accelerated tasks where both runtimes are bottlenecked by WebGPU compute shader overhead rather than kernel quality. The honest caveat is that the benchmark hardware is aspirational for most web deployment contexts - a 2024 M4 is not the median user device, and the 5-60x GPU figure needs field data from a wider hardware distribution before it informs deployment decisions. The WebNN path is worth watching: when NPU access through the browser becomes stable and broadly available, it will make certain embedded ML use cases - real-time audio classification, wake-word detection, continuous gesture recognition - viable in a browser context that currently requires a native app. That is still a feature flag experiment, not a production reality. For practitioners today, the practical entry point is XNNPACK CPU inference for models under 50MB, with WebGPU as an opt-in acceleration layer for users on capable hardware.

Discussion Question

LiteRT.js makes it technically feasible to run classification, detection, and embedding models entirely client-side with no data leaving the browser. For automation practitioners building tools that handle sensitive documents - tax forms, medical records, legal contracts - the privacy argument for in-browser inference is strong. But client-side models are also inspectable and extractable by users with browser developer tools. What is your threshold for accepting model IP exposure risk in exchange for the privacy and latency benefits of running inference in the browser?