Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

agent-desktop

AI overview

Sign in and the AI will write an overview from our coverage.

Headlines · 3

  1. GUI Agents Face Reliability Limits in the Accessibility Layer

    A report argues that screenshot-free GUI agents depend on stable, uniquely named nodes in the operating system’s accessibility tree. Its audit of two interface forms found unnamed-node rates of 0% and 50%. In a simulated tree change, 70% of reference positions pointed to different controls. The report also notes that agent-desktop’s reported 55 consecutive successful actions correspond to a 6.5% upper bound on the failure rate at 95% confidence under the stated statistical assumptions, rather than establishing a failure rate below 1%.

    掘金 · 人工智能 · 🔥 0
  2. agent-desktop 1.2.0 adds MCP support for background desktop control

    Vercel Labs released agent-desktop 1.2.0 with MCP support, according to 掘金 · 人工智能. The tool reads the operating system accessibility tree and assigns numbered UI controls so an AI can trigger them without screenshots or taking over the mouse and keyboard. In the author's WorkBuddy test, it made 55 error-free Calculator clicks and recorded a total of 1746 in Notepad, although actions were slow, consumed credits per step, and could not handle every interface in the background.

    掘金 · 人工智能 · 🔥 0
  3. Vercel Labs Ships agent-desktop for Cross-Platform Computer Use

    Vercel Labs has introduced agent-desktop, a CLI and C library that lets AI agents inspect and operate desktop applications through operating-system accessibility trees on macOS, Windows, and Linux. The project documentation says agents can receive actionable JSON references, click or type against them, and re-check the interface, while the tool itself contains no model. It also warns that snapshots, clipboard reads, screenshots, command output, and debug artifacts may expose private desktop content.

    GitHub Trending(每日) · 🔥 0

Experience and discussion from the community

Share my agent-desktop experienceAsk about agent-desktop

agent-desktop: Computer use for AI agents through the operating system accessibility tree.

What problem it solves AI agents need a way to observe and operate desktop applications through their native user interfaces. agent-desktop provides this interaction model by reading the accessibility tree, returning actionable references in JSON, and allowing agents to verify the result by reading the interface again. Key features Accessibility-tree snapshots with actionable references — Snapshot…

github.comhttps://github.com/vercel-labs/agent-desktop