Blazing fast on-device GenAI with LiteRT-LM

Source: Google Developers Blog

Overview

Google AI Edge’s LiteRT‑LM provides a production‑proven, highly optimized infrastructure for running Gemma 4 across cross‑platform mobile and edge environments. It actively unlocks the model’s native multimodal and agentic features on‑device by utilizing memory‑efficient dynamic loading, Multi‑Token Prediction for up to a 2.2× speedup, and advanced orchestration tools like Thinking Mode and Constrained Decoding.

Furthermore, the engine is rapidly expanding its integration surfaces beyond Android, introducing new native Swift APIs for Apple ecosystems and WebGPU‑accelerated JavaScript APIs for high‑performance, serverless browser inference.