AI Model Generates Millions of Lines of Proof Code to Verify Software Safety

Anthropic has announced that its Claude AI model can now automatically generate formal mathematical proofs to verify the correctness of existing software. The development, achieved in partnership with Theorem, represents a significant step toward automating one of the most rigorous aspects of software engineering. By producing these proofs, the AI system addresses a long-standing challenge in the industry: ensuring that code behaves as intended without requiring extensive, manual review by specialized teams.

The scale of the operation was substantial. Over a period of just a few weeks, Claude generated approximately 180 million lines of proof code. This work focused on 123 different open-source libraries sourced from Google’s OSS-Fuzz security project. The model did not write new software; instead, it analyzed existing compiled machine code and translated crash tests into mathematical problems to demonstrate their correctness. This approach allows for a deeper level of assurance than traditional testing methods, which often fail to cover every possible scenario.

The effort was led by Anthropic’s Frontier Red Team in collaboration with Theorem. The process was described as "fully autonomous," meaning it required minimal human intervention during the generation phase. However, the underlying formal foundations and tools were developed by human experts at Theorem. An independent verification program named Rocq automatically checked the generated proofs. According to Anthropic, this automated checking process eliminates the need for large teams of human reviewers, a stark contrast to previous methods where formal verification was a labor-intensive endeavor.

Historically, formal verification has been reserved for critical systems due to its high cost and complexity. For example, a team of specialists required 20 person-years to formally verify the seL4 operating system kernel. This made the technique impractical for most software, limiting its use to high-stakes environments like avionics. By leveraging AI to handle the proof generation, Anthropic and Theorem have significantly reduced the time and resource barriers associated with this level of scrutiny.

A key outcome of this project was the discovery of hundreds of bugs that had been missed by conventional testing. These flaws often only manifested under specific conditions, such as unusually large input files or particular sequences of data access. In one notable instance, the team found as many security bugs in a single week within a widely used PDF reader as are typically reported for that software in an entire year. In another case, Claude identified a logical error that had remained undetected for more than a decade.

Despite the successes, the project comes with strict practical limitations. Anthropic notes that "formally verified" does not equate to "error-free." The proofs are valid only under specific conditions, including a particular processor architecture, single-threaded execution, and inputs under 64 MB. In real-world scenarios, these libraries often run on diverse hardware and configurations, meaning the verification may not cover all operational contexts. Additionally, the computational cost is significant, with individual proofs consuming hundreds of millions of tokens.

The project also highlights improvements in model efficiency. A smaller test run demonstrated that proofs are reusable across different contexts. Furthermore, newer models are becoming more efficient; Opus 5.5 solved half of the proofs with approximately 40 percent fewer tokens than its predecessor, Opus 5. This efficiency gain suggests that the cost of formal verification may decrease as AI models continue to improve.

The implications of this work extend beyond immediate cost savings. If AI can both write code and prove its correctness under defined conditions, the industry’s focus may shift from asking whether AI can program to whether it can guarantee safety. Anthropic expects formal verification using AI to become as routine as standard testing in the near future. Theorem’s roadmap already targets critical infrastructure components such as glibc, OpenSSL, and cURL, indicating a plan to expand this approach to foundational software systems.