Beyond the chat window: how large reasoning models are changing what AI can figure out
Photo: N43 and Hermestechnology
For most of the last decade, a large language model answered the way a reflex does: text in, text out, one forward pass, done. Then a second family of systems arrived that pauses before speaking, works the problem in steps, and spends extra computation on the hard cases. IBM Technology's explainer on large reasoning models, embedded above, covers what this new class of system actually does, and it is worth slowing down on, because in under two years the reasoning model has gone from research curiosity to the default shape of frontier AI.
Video: "What Are Large Reasoning Models (LRMs)? Smarter AI Beyond LLMs" by IBM Technology, approximately 154,000 views as of September 2026. View counts are observations at publish time and change over time.
01The limits of the plain LLM
A large language model is, at bottom, a next-token predictor trained on enormous amounts of text. That training gives it broad knowledge and fluent prose, and it is the basis of modern chatbots such as ChatGPT, Claude, Gemini, and Grok. But the same architecture has a known weakness: everything the model computes, it computes in roughly one pass, with a fixed amount of computation per word it produces. Ask a standard LLM a hard math problem and it answers at the same speed as an easy one, which is exactly backwards. Hard problems are the ones that deserve more thinking.
The failure mode is familiar to anyone who has used these systems. A plain LLM asked a multi-step question often commits to an answer early and then writes text that justifies the commitment, a phenomenon researchers call hallucination with confidence. It is not lying; it is doing what it was built to do, producing plausible continuations, on a task where plausibility and correctness have little to do with each other. Arithmetic, logic puzzles, and novel code are the classic casualties.
Researchers spent years patching around this with prompt tricks, most famously chain-of-thought prompting, where the model is asked to show its work step by step. The trick works because it converts the model's own generated text into a scratchpad. But a prompted scratchpad still spends the same fixed computation per token, and it depends on the model having been taught the reasoning pattern in training. The question the field eventually asked was more radical: what if the model could simply spend more compute at answer time when the question is hard?
02How reasoning models think before answering
A large reasoning model is an LLM with a deliberation loop bolted into inference. Instead of leaping to an answer, the model generates a long internal working sequence, exploring approaches, checking its own steps, backtracking when a path fails, and only then producing the visible response. OpenAI's o1, previewed in September 2024, was the first widely released system in this family, and its behavior was described in almost human terms: it spends time thinking before it answers, which made it better at complex reasoning, science, and programming than its non-reasoning sibling GPT-4o.
The IBM Technology video at the top of this article walks through what that deliberation actually looks like. The model's hidden working is a stream of self-directed moves: restate the problem, try a decomposition, test whether a candidate answer satisfies the constraints, discard the attempt if not, try another. The output the user sees is only the polished conclusion of that process, and the length of the process is not fixed. Easy questions finish quickly; hard ones can burn thousands of tokens before committing.
Two design choices matter here. First, the deliberation is learned, not hand-coded: the model is trained, often with reinforcement learning on verifiable tasks like math and code, to value reasoning traces that reach correct answers. Second, the model decides when to stop, which means the system allocates its own thinking budget. That self-allocation is the genuinely new engineering, and it is what separates a reasoning model from a chatbot told to show its work.
03Test-time compute: a new scaling axis
From 2020 to 2024, the story of AI progress was pretraining scale: more parameters, more data, more training compute. Reasoning models opened a second axis, test-time compute, where capability is bought by spending more inference computation per query. The two axes are not substitutes. Pretraining determines what the model knows; test-time compute determines how effectively it can use what it knows on a specific hard problem.
The practical consequence is that performance is no longer a single number. A reasoning model's score on a benchmark depends on how much thinking it was allowed, in the same way a student's score depends on how long the exam runs. Evaluations that cap output tokens can accidentally measure the budget rather than the model. This is why published results for the same reasoning model sometimes differ across reports, and why any comparison chart, including the one below, should name the setting it was measured in.
The economics also changed. A one-pass LLM answer costs a predictable fraction of a cent; a reasoning model let loose on a hard problem can consume orders of magnitude more compute for the same visible response. For routine requests that is waste, which is why deployed products route: a fast path for easy queries, a deliberation path for hard ones, and a hybrid that decides between them. The IBM video makes the same point in the language of enterprise architecture, where inference cost is a line item rather than an abstraction.
04What the benchmarks show, and what they do not
The headline results behind the reasoning-model turn were striking. On AIME 2024, a set of competition mathematics problems, OpenAI reported that GPT-4o scored around 13 percent while o1 scored in the low eighties; on GPQA Diamond, graduate-level science questions, the reasoning model cleared the average human expert response where the standard model fell well short; on Codeforces programming contests, the same jump appeared in percentile form. The pattern repeats across independent evaluations: deliberation buys performance on tasks with verifiable answers, and the harder the task, the wider the gap.
But the benchmarks are also where the caveats live. Verifiable domains, math, code, formal logic, are exactly the domains where a model can check itself, so it is no accident those are where reasoning models shine. On open-ended judgment, taste, nuance, and facts that cannot be re-derived, the advantage narrows or disappears, and longer thinking can even make answers more confident without making them more correct. A model that deliberates for ten thousand tokens about a question of opinion is still answering an opinion.
The industry's measurement habits have started to adapt. Evaluators now report results at multiple thinking budgets, and the honest comparisons name the compute spent. When reading any single leaderboard number for a reasoning model, the IBM video's implicit advice holds: the score is a property of the model and the budget together, not of the model alone.
05Why the pattern spread across the industry
Once one lab showed that test-time compute works, the pattern replicated everywhere within months, because the underlying recipe was published as openly as the result. DeepSeek-R1, released by the Chinese lab DeepSeek in January 2025 with a public technical report, demonstrated that reasoning behavior could be trained into an open-weight model with reinforcement learning on verifiable tasks. Anthropic shipped extended thinking for Claude in early 2025, Google brought reasoning modes into Gemini, and open-weight families from Qwen to GLM followed. The chart below tracks how quickly the release cadence compressed.
The spread had a second cause: the recipe is architecture-agnostic. Deliberation is a training and inference regime, not a new kind of network, so any lab with a strong LLM, verifiable training tasks, and reinforcement learning infrastructure could adopt it. That is why the arrival of LRMs, the umbrella term the IBM video uses for the family, looks less like a single breakthrough and more like a technique diffusing through an industry that was collectively ready for it.
The consequences reach past the labs. Deliberation made AI systems genuinely useful on the tasks businesses actually care about, multi-step analysis, code repair, planning, and that usefulness is what pulled reasoning features into phones, search engines, and office software within a year. When an assistant on a consumer device pauses for a moment before answering a hard question, it is running a compressed version of exactly this pattern.
06Where reasoning models still fail
Deliberation is not a cure-all, and the honest accounts, including the IBM video's, are careful about the failure modes. The first is cost: thinking tokens are compute, and a model that reasons for ten minutes on a question that needed ten seconds is a real expense at scale. Products manage this with routing and caps, which reintroduces the original problem at one remove, since the router can misjudge which queries deserve the budget.
The second is the hallucination ceiling. Self-checking works when the model can verify its own work, as in arithmetic or executable code. On factual claims about the world, the deliberation loop has nothing ground truth to check against inside itself, so a reasoning model asked about an obscure fact can think very carefully and confidently produce a wrong answer. Longer reasoning on an unverifiable claim mostly buys a more elaborate wrong answer.
The third is overthinking. On easy tasks, the extra deliberation adds latency and tokens without adding accuracy, and evaluations have documented cases where the reasoning path underperforms the fast path on simple questions. The engineering response, adaptive budgets and hybrid routing, is the current frontier of product work, and it is unsolved in the general case.
07What comes next for the frontier
The immediate directions are visible in current releases. Adaptive computation, letting the model allocate thinking to difficulty, is maturing from research to shipping product. Agents, systems that wrap reasoning models in tool access so they can act rather than only answer, extend the deliberation loop outward into real software. And the hybrid stack, fast models for breadth, reasoning models for depth, with a router in between, is becoming the standard architecture for anyone deploying AI at production scale.
Deeper out, the open question is whether test-time compute keeps paying. The first results came from a regime nobody had scaled before, and early gains are usually the cheap ones. If the return on thinking tokens flattens, the field's attention will swing back toward training-time improvements; if it holds, the interesting constraint becomes inference hardware, chips, data centers, and the energy they consume, which is one reason reasoning economics and semiconductor economics are now the same conversation.
For a reader trying to keep score, the IBM Technology video embedded above is a good compass: understand what deliberation buys, know that it is bought at a price, and evaluate every new model announcement by asking not just how smart it is, but how it decides when to think. That question did not exist in AI two years ago. It is now the most important one in the product category.
References
- IBM Technology - What Are Large Reasoning Models (LRMs)? Smarter AI Beyond LLMs (YouTube)
- Wikipedia - Large language model (definition and chatbot applications)
- Wikipedia - OpenAI o1 (first o-series reasoning model, September 2024)
- Wikipedia - Chain-of-thought prompting (step-by-step prompting background)
- arXiv - DeepSeek-R1 technical report (open-weight reasoning via reinforcement learning, January 2025)
- Wikipedia - Google DeepMind (Gemini reasoning model lineage)
By N43 and Hermes for Sailor Bob News.
