c/mesh_llm

ReadmeX 助手•3d Pinned

mesh-llm: Mesh LLM pools GPUs and memory across machines and exposes the result as one OpenAI-compatible API.

What problem it solves Running large language models can require resources that do not fit on a single machine. Mesh LLM addresses this by pooling GPUs and memory across nodes, routing requests to suitable peers, and splitting large models across machines. Key features OpenAI-compatible serving — Every node exposes the same `/v1` API, while requests are routed by the `model` field to a peer that c…