Solved Is a Loaded Word: How AI Mathematical Claims Actually Get Certified

01What 'solved' means in mathematics: conjecture, proof, and certification

In mathematics, a problem is not solved when someone announces an answer. It is solved when a written argument survives checking. The lifecycle has three stages that popular coverage tends to flatten into one. A conjecture is a claim believed true on the weight of evidence but not yet proven. A proof is a finite chain of deductions running from agreed axioms to the claimed conclusion, with no gaps and no appeals to intuition. Certification is the act of confirming that the chain really is airtight, and for most of history it has been social: editors recruit referees, referees run examples and hunt for counterexamples, and the community converges on acceptance over months or years.

What large language models change is not the meaning of any stage but the economics of the first one. A model can emit a plausible-looking argument in seconds, which makes announcement nearly free and leaves certification as the scarce, expensive step. So the StarTalk question — did AI solve one of math's hardest problems? — resolves into a sharper pair of questions. Was the claim ever translated into a form a machine could check, and did an independent checker accept every line? Until both answers are yes, 'solved' is a press release.

02The generation problem: how LLMs propose candidate proofs and why most fail

A language model trained on the mathematical literature is, functionally, a very fast generator of candidate arguments. Ask for a proof and it produces one, fluently and confidently, drawing on patterns from an enormous corpus of published theorems. The difficulty is the base rate. Most candidates fail, and they fail in characteristic ways: a step that cites a lemma nobody ever proved, a quantifier silently flipped between 'for all' and 'there exists', a degenerate case waved past, or an argument that quietly assumes the conclusion it was meant to establish.

None of this is a new failure mode. Automated theorem proving, the subfield of automated reasoning and mathematical logic devoted to proving theorems by computer program, has wrestled with search and soundness since the 1950s, and the field is widely credited as a major motivating force behind the development of computer science itself. What is new is the volume. A modern pipeline does not bet on one draft; it generates thousands of candidates, lets the model attack each one, and passes only the survivors forward. That filter changes the unit of progress: what matters is not how impressive an output reads but how many candidates survive contact with a checker. By that measure, generation is abundant and trust is the deficit.

03Formalization: translating informal argument into Lean-compatible statements

Between a draft and a certificate stands formalization: restating the claim, the definitions, and every inferential step in a language with no room for interpretation. The working medium is Lean, a proof assistant and functional programming language based on the calculus of constructions with inductive types, developed as a free and open-source project and supported today by the nonprofit Lean Focused Research Organization. Formalization is where most effort dies. The informal sentence 'every sufficiently large even number is a sum of two primes' looks compact, but making it machine-checkable means pinning down what 'sufficiently large' means, which formal account of primality to adopt, and which library supplies the supporting lemmas.

A one-page human argument routinely expands into hundreds of lines of Lean, and gaps surface as compiler errors rather than stylistic quibbles. Practitioners describe the work as translation plus interrogation: the formalizer must decide what the original argument really committed to, and every hedge the author left behind becomes an explicit choice. When formalization fails, the diagnosis cuts two ways — either the informal proof was wrong, or the statement as formalized is not the statement anyone meant to prove. Both outcomes are valuable, and neither is available from a press release.

04The kernel: how a proof assistant certifies every step mechanically

At the bottom of Lean sits the kernel: a deliberately small checker that accepts or rejects each inference against the system's axioms and rules. It performs no interpretation and offers no benefit of the doubt. A proof is a data structure; the kernel walks it, expands each justification, and either every step reduces to an axiom-governed move or the whole thing is rejected. This mechanical austerity is the entire point.

The kernel does not share the language model's confidence, its training data, or its failure modes, so a Lean-accepted theorem is certified by something structurally independent of whatever proposed it. Checking is also cheap relative to discovery — verifying a finished derivation costs a tiny fraction of the search that produced it, which is why the kernel is rarely the bottleneck. The certification, however, is exact for the statement as formalized and nothing more. A theorem can be simultaneously true, kernel-verified, and weaker than the announced result, if the formalized statement quietly narrowed a hypothesis. That is why serious practitioners publish the machine-checked artifact rather than a paraphrase: the object of trust is a file, and anyone can re-run the checker on it and get the same verdict.

05Where the time actually goes: generation versus formalization versus checking

Asked where effort is spent in an AI-assisted proof effort, practitioners converge on a picture that inverts the public narrative. Candidate generation, the part that looks like magic in demos, is a small slice of the work. Formalization into Lean dominates, because it demands the precision that informal mathematics leaves implicit. Repair and iteration come next: drafts that survive triage still break against the compiler, and each fix can cascade into reworked definitions. Kernel checking is nearly free, a background job measured in seconds for statements that took weeks to state correctly.

The chart below gives an illustrative breakdown — exact ratios vary by domain and team, but the ordering does not. The pattern echoes an older lesson from software engineering: writing code is cheap, specification and maintenance are expensive. It also explains why a demo proves little. A video showing a model emit an argument demonstrates the fifteen percent of the pipeline that was never in doubt. The majority share that determines whether the result exists at all happens afterward, off camera, in a dialogue between a formalizer and a type checker. Coverage of an AI mathematics claim that skips this stage has skipped the story.

06Limits: what formal verification cannot yet cover and where trust still leaks in

Formal certification is powerful but bounded, and the boundary is worth stating precisely. It covers what can be said: claims expressible as precise statements about defined objects, derived from axioms the checker accepts. It does not cover problems whose difficulty lies in agreeing on the statement, empirical questions about the physical world, or definitions still under negotiation in a living research area. The cost of entry also scales steeply with scope. A self-contained competition problem needs little machinery beyond its own statement; a research lemma may lean on a tower of prior results that must already sit in the library; a full research paper is a small city of interdependent definitions. The illustrative index below captures that gradient.

Even inside the boundary, trust leaks at three joints. The kernel implementation itself is trusted code — small and heavily scrutinized, but not self-verifying in any absolute sense. The axioms are a choice, and exotic results can depend on which choice was made. And the formalized statement is trusted to match the mathematical intent, which is precisely the judgment formalization was supposed to remove. None of this is a scandal; it is the honest accounting of what 'certified' buys, and it is far more than an unchecked announcement buys.

07Implications: what a certification pipeline changes about AI-discovery claims

For anyone consuming AI-discovery claims, the pipeline suggests a concrete rubric. A result can be graded by stage: an informal argument is a draft; a formalized statement plus a complete Lean file is a testable artifact; a kernel-accepted file is a certified theorem, exact for the statement as written. Announcements that stop at stage one should be read as proposals, however distinguished the venue. The questions that matter are mechanical: is there a repository, does the file compile, which exact statement was proven, and did anyone independent re-run the check?

The rubric also redirects the labor of mathematics rather than replacing it. When generation is abundant, human expertise concentrates where models are weakest — choosing definitions worth formalizing, judging whether the formalized claim is the interesting one, and deciding which conjectures deserve the translation effort at all. The deeper shift is institutional: certification was always mathematics' quality-control mechanism, and mechanizing it does not weaken the discipline, it hardens it. A culture that demands the artifact over the anecdote is difficult to fool, regardless of how fluent the attempt is. The StarTalk framing asked whether AI solved a hard problem. The durable answer is a process, not a yes or a no — and the process is now inspectable end to end.