AI brings formal verification into the mainstream, with caveats
An essay argues that AI has pushed formal verification into the mainstream, citing growing startup activity, a billion lines of Rocq generated for one project, and work by Theorem with Anthropic and Harmonic on software and hardware verification. It cautions that “formally verified” systems can still contain bugs because specifications, models, provers, compilers, and machine semantics may remain trusted or imperfect. The author predicts binary-level proof generation for some first-order properties of single-threaded programs, while concurrent, distributed, higher-order, and non-functional properties will require years more work.
The essay presents a substantive view of where AI-assisted formal verification is becoming practical while explaining why broad claims of verified or bug-free software remain misleading.
AI is making formal verification go mainstream...
...I wrote in December 2025, well, after the last 12 months I believe it is hard to argue otherwise. AI has made formal verification go mainstream.
Not only did the number of formal verification startups skyrocket, Claude also wrote a billion lines of Rocq for a single verification project, everyone from the creator of Claude Code to the Vercel CEO, lead Codex/Cursor developers is talking about formal verification. The early startups have started to show economic value in software on top of simply producing math proofs: Theorem's binary verification work with Anthropic, Harmonic verifying C programs and hardware designs... With the cybersecurity risk going through the roof, formal verification is not just easier, it is also getting more economically valuable.
The questions people ask now are not "how can we prove X", but rather "what can we prove about X"; or "what is left that we can't prove"? These are good questions to ask, I am writing this post in part to answer them; and in part as a follow-up to last year's post "Test, don't (just) verify". To tease the ending, I am very optimistic about formal verification, but I think people mean lots of different things when they say something is "formally verified", which I will try to break down into the parts I'm optimistic and pessimistic about.
#Anthropic#Claude#Claude Code#Theorem#Harmonic#Google DeepMind#Signal#Rocq#Lean