vllm-ascend: A community-maintained hardware plugin for running vLLM on Huawei Ascend NPUs.
What problem it solves vLLM users need a way to run the framework on Huawei Ascend NPU hardware through a supported integration path. The plugin decouples Ascend NPU integration from vLLM and enables supported model types to run on the platform. Key features Hardware-pluggable vLLM integration — The plugin provides an interface for integrating Ascend NPUs with vLLM and follows the community's hard…
github.comhttps://github.com/vllm-project/vllm-ascend