Diagnosing silent CPU fallback on RK3588 with RKNN PERF data
A Juejin article documents how to identify operators silently falling back from the RK3588 NPU to the CPU despite correct inference results. The author uses librknnrt's rknn_query(RKNN_QUERY_PERF_DETAIL) output, enabled with RKNN_FLAG_COLLECT_PERF_MASK, and classifies execution by CPU time share. On-device tests of YOLOv8n and YOLOv8s-seg were classified as NPU_ONLY, while the thresholds and broader conclusions remain the author's own methodology.
The workflow gives edge-AI developers a concrete way to distinguish genuine NPU execution from silent CPU fallback in modified RKNN models, rather than relying on correct outputs or speed alone.
先说一个反直觉的结论
模型在 RK3588 上推理出了正确结果，不代表它在 NPU 上跑。
算子不被 NPU 支持时，RKNN runtime 会静默回退到 CPU 执行：程序不报错、结果是对的、只有速度慢得莫名其妙。更麻烦的是，官方模型仓库（model zoo）里的模型一般都很干净，真正会回退的恰恰是大家自己改过结构、加过算子的模型——网上现成的经验帖帮不上忙，只能自己查。
这篇写我实际用的查法：不靠估算，直接调 librknnrt 的 PERF 接口读硬件侧的逐算子账本，再按 CPU 时间占比给结论。包含一个差点让我得出错误结论的坑，和两个模型的真机实测数据。测试环境：RK3588，Ubuntu 22.04，rknpu 驱动 0.9.8，librknnrt 2.3.2。
第一步：坑——PERF 数据会静默为空
查回退的正道是 rknn_query(RKNN_QUERY_PERF_DETAIL)，它返回每个算子的耗时明细。但有个巨坑：
#RK3588#RKNN runtime#librknnrt#rk-edge-doctor#YOLOv8n#YOLOv8s-seg