创建

登录 ReadmeX

或

了解我们如何处理你的信息: 隐私政策

ReadmeX

陪你读懂 AI 动态

今天,想先了解什么?

选一份动态总结,或直接聊聊你的问题。

社区和关注动态总结只属于你自己。

资讯

ExecuTorch WebGPU让PyTorch模型在浏览器运行

AI 总结

PyTorch Blog介绍了ExecuTorch WebGPU后端,导出的PyTorch模型无需转换为GGUF、ONNX或TFLite格式,即可在支持WebGPU的浏览器中本地运行。同一个.pte文件可运行于Chrome和Safari,并覆盖语言、视觉语言、视觉、语音及端侧训练工作负载,提示词和其他数据均保留在设备上。PyTorch称,在Chrome和Apple M1 Pro上使用量化版Llama 3.2 1B与Qwen3-0.6B测试时,WebGPU的prefill速度为WebAssembly CPU路径的4至7倍,decode速度为2至10倍。

深度见解

该后端为PyTorch开发者提供了通过同一模型文件在浏览器及多平台部署本地GPU推理的直接路径,但所报告的性能数据来自特定环境下的厂商基准测试。

TL;DR

  • PyTorch models now run in the web browser on the GPU through ExecuTorch, with no conversion to another framework’s format. You export with torch.export to an ExecuTorch .pte file, and the same file runs in any WebGPU browser via ExecuTorch’s new WebGPU backend.
  • It works across a broad range of models: LLMs (e.g. Llama 3.2, Qwen3, Gemma 3), vision-language (e.g. SmolVLM), segmentation, depth, object detection, super-resolution and speech-to-text (e.g. Whisper). Everything runs locally in the tab, and no data leaves the device. Try it in the ExecuTorch WebGPU Playground.
  • On the same machine, WebGPU is 4–7× faster than ExecuTorch’s CPU path (Webassembly + XNNPACK) on prefill and 2–10× faster on decode.

Why the Browser?

The web is the largest deployment surface with billions of users on Chrome, Firefox and Safari. Browser inference ensures privacy by keeping users’ prompts, model inputs, and outputs on the user’s device rather than the server. This reduces network latency, enables offline use-cases, and lowers server-side inference costs. With the ExecuTorch WebGPU backend, PyTorch models get a native path to GPU-accelerated inference in WebGPU-enabled browsers on desktop, laptop and mobile.

#PyTorch#ExecuTorch WebGPU backend#WebGPU#Llama 3.2 1B#Qwen3-0.6B#Qwen2.5-0.5B#Gemma 3 1B#SmolVLM#Whisper#Depth Anything V2#SAM 2#D-FINE

原文约 4 分钟读完阅读原文
群聊

还没有评论,来说说你的看法。