Developer open-sources moray-ghost local GUI agent engine
The developer of moray-ghost says it is an open-source local GUI agent execution engine with local vision, multithreading, Windows input handling, MCP support, and safety controls. It reportedly uses OCR and OpenCV to create a JSON directory of interface controls, allowing a model to select a control ID while the engine maps it to screen coordinates; the developer also demonstrated it drawing in Windows Paint with a physical mouse. Performance, precision, and production-readiness claims have not been independently verified.
The project offers a concrete alternative to screenshot-and-coordinate GUI workflows by separating control identification from coordinate calculation and adding execution safeguards, though its real-world effectiveness remains self-reported.
在体验了市面上的云端 Computer Use 后，我总觉得差点意思：延迟高（每步 3-5 秒）、按 Token 极度烧钱、大模型经常产生坐标幻觉瞎点，而且极度不安全。
为了打造一个真正能“安全接管电脑”的 AI，我爆肝了几个日夜，从零手搓了一个带本地视觉、多线程并发、军工级防呆的底层执行引擎：moray-ghost。
👇 先看一个震撼的实操：它用真实的物理鼠标，在 Windows 画图里一笔一划复刻出了《DeepSeek 鲸鱼娘》
💥 一、 我们如何打穿现有 Agent 框架的痛点？
现有的 GUI 框架大多停留在“截图 -> 发给大模型 -> 返回坐标 -> PyAutoGUI 点击”的原始阶段，在真实生产中完全不可用。moray-ghost 的解法是：