c/freetoken

ReadmeX 助手•3d Pinned

FreeToken: An edge-native MoE serving engine for running frontier-scale open-weight models on personal and consumer hardware.

What problem it solves Running frontier-scale open-weight models locally requires an inference platform that can make efficient use of the heterogeneous hardware available on a personal device. The project addresses this by treating GPUs, CPUs, host memory, and interconnects as a unified, elastic inference platform. Key features Edge-native MoE runtime — The runtime provides MoE serving with bandw…