How to Assess Lean Verification of AI-Generated Proofs
An analysis argues that successful Lean compilation is only one part of assessing an AI-generated proof. Its four-part framework asks whether the Lean statement matches the paper, whether the code compiles, whether the proof uses unintended axioms, and whether it exploits bugs in Lean’s kernel. Even if all checks pass, the Lean proof may differ from the proof in the paper’s PDF. The analysis reports that OpenAI’s theorem dump contained 722 papers, around 45% with some Lean code, and that about 38 of 290 corresponding entries did not state, let alone prove, the papers’ main theorems. The author also says these checks have not been systematically applied to many papers in the dump and sees no reason to believe they will fail.
The framework provides concrete checks for interpreting claims that an AI-generated formal proof is correct. It also shows why a verified Lean proof does not by itself establish that the corresponding paper contains the same or a correct proof.
This post is an attempt to explain in simple terms how to interpret claims of the form “this theorem is correct because it was checked in Lean”. The summary is: there is an important signal that can be extracted from Lean code, but if this code is AI-generated then you need to do some tests first. We explain the tests, and note that in many cases (such as the OpenAI theorem dump) these tests have not yet been systematically done.
Introduction
Last month’s theorem dump by OpenAI consisted of 722 mathematics papers, around 45% of which came with some Lean code. Three of the (non-Leanified) papers have since been retracted because of a sign error. Some people might say “well, the papers that come with Lean code are definitely correct because that’s how Lean works”. Others might say “well Lean had a soundness bug and could well have more, and AIs will reward hack, so Lean verification doesn’t tell us much at all”. The truth, unsurprisingly, is more subtle, and this post is an attempt to explain it.