Google open-sources ML Drift, a cross-platform GPU inference engine for on-device AI
Google's AI Edge team released ML Drift under Apache 2.0, a cross-platform on-device GPU compute engine for AI/ML inference that abstracts OpenGL ES, OpenCL, Metal and WebGPU backends and serves as the GPU acceleration engine inside LiteRT, while also being usable standalone. The release adds 5D tensor support, an extensible custom-op framework with an agent-oriented SKILL.md guide, and stage-aware optimizations for on-device LLM prefill and decode; a WebGPU backend built on Dawn extends it to Windows and Linux as a desktop preview. Google says ML Drift already runs on millions of devices powering Chrome, YouTube Shorts, Photos and Meet, with YouTube Shorts seeing up to 40% lower average frame latency, and partners including Adobe and Snap reporting up to 30% faster on-device performance — all vendor-reported figures.
Fragmented edge GPU architectures, drivers and low-level APIs are a major obstacle to local generative AI; a cross-platform open-source engine could cut the cost of porting and optimizing for it.