vllm-ascend: A community-maintained hardware plugin for running vLLM on Huawei Ascend NPUs.
What problem it solves
vLLM users need a way to run the framework on Huawei Ascend NPU hardware through a supported integration path. The plugin decouples Ascend NPU integration from vLLM and enables supported model types to run on the platform.
Key features
Hardware-pluggable vLLM integration — The plugin provides an interface for integrating Ascend NPUs with vLLM and follows the community's hardware-pluggable approach.
Source: README
Support for multiple model types — Transformer-like, Mixture-of-Experts, Embedding, and multi-modal LLM workflows can run through the plugin on Ascend NPUs.
Source: README
Version-aligned release branches — The project maintains a main branch and release branches corresponding to specific vLLM versions, with CI coverage for supported branches.
Source: README, release_notes
Community contribution and support — Developers can access contribution guidance, report bugs, and ask usage questions through the project's documented community channels.
Source: README, CONTRIBUTING.md
Who it may suit
Inference: This project may suit vLLM users who need to deploy supported models on Huawei Ascend NPUs, as well as contributors working on Ascend backend integration.
Topics to discuss
Which model and hardware combinations should receive priority in future releases?
How can the plugin make installation and troubleshooting easier for new users?
What contributions or feedback would be most useful for the Ascend hardware integration community?
Written by ReadmeX AI from the project's README, source and public GitHub data; it may contain mistakes.
Read the original README https://github.com/vllm-project/vllm-ascend#readme
ReadmeX opened this community for a trending project; it is not run by the project. Maintainers can claim it from the community page.
Comments
…