How AI agents actually work: the anatomy of autonomous software

01 What an AI agent is, and what it is not

An AI agent is a system in which a language model does more than answer: it decides what to do next, invokes external tools to do it, observes the result, and continues until it judges the task complete. The distinction from a chatbot is agency over sequence. A chatbot answers one prompt and stops; an agent plans across many steps, chooses among actions, and reacts to what its actions cause. The word autonomous refers to that loop, not to magic: every step inside it is a model call, a function execution, or a database write, all of it ordinary software.

The confusion is understandable because the term is applied to everything from a script with an API call to research prototypes that plan for hours. The useful definition sits in the middle: tool access, multi-step planning, persistence across steps, and some stopping criterion. Systems missing one of those pieces are better described as automations or assistants. As the Tech With Tim build walkthrough referenced below demonstrates by construction, an agent is less a product feature than an architecture assembled from parts, each of which can be built well or badly.

02 The core loop: model, tools, and memory

Every agent rests on the same cycle: perceive the current state, reason about it, act through a tool, and observe the result before reasoning again. The chart above sketches where the time goes in a typical iteration, and the shape is lopsided for a reason: the model call, the reasoning step, dominates by an order of magnitude. Tool calls finish in tens of milliseconds; deciding what to do next takes hundreds. This asymmetry drives agent economics, since an agent that reasons for forty steps pays forty model calls even when every action itself is trivial.

Memory is the second structural piece, and it comes in layers. Working memory is the context window: everything the model can attend to in the current call, including the task, its plan, and the recent history of actions and results. Persistent memory lives outside the model, in files, vector databases, or task state that survives between calls and sessions. The distinction matters because context windows, though now enormous, are not infinite, and the engineering of what to keep, summarize, and retrieve on demand is where many agent projects succeed or fail in practice.

03 Planning and task decomposition

Given a goal such as research a market and produce a report, the model first decomposes it into subtasks, then executes each, revising the plan as results arrive. Modern implementations mostly use the model itself as the planner, prompting it to emit an explicit plan and then re-plan after each observation, an approach that trades some efficiency for adaptability. The alternative, fixed workflow graphs with model calls at specific nodes, gives up autonomy in exchange for predictability, and production systems frequently choose exactly that trade.

Planning is also where an agent's intelligence is most visible. A capable planner sequences dependencies correctly, holds long-term constraints such as budget or deadline in view, and abandons failing branches early. A weak planner thrashes: it re-derives the same subgoal, contradicts its earlier decisions, or marks tasks complete that it never performed. Evaluating planning quality, not just answer quality, is the hard part of agent benchmarking, and it is why the same model can look brilliant in a demo and clumsy in a workday.

04 Tool use and function calling in practice

Tools are the agent's hands: search APIs, code interpreters, file systems, databases, email, payment rails. Structured tool use works through function calling, in which the developer describes each available function with a name, schema, and purpose, and the model responds not with prose but with a structured request to invoke one. The runtime executes it and returns the output as an observation the model consumes in its next turn. The pattern is deliberately boring engineering, and its reliability is what makes the rest of the agent stack credible.

Designing the tool surface is an art with real consequences. Too few tools and the agent cannot act; too many and the model's choice degrades as descriptions compete for attention. Successful designs give the agent a small, composable toolkit, make failure states explicit so the model can recover, and log every invocation for audit. The walk-through referenced below builds this layer in real time, and the construction makes the central point visible: an agent is only as good as the interface between reasoning and action, which is human-designed all the way down.

05 Failure modes: loops, hallucinated actions, and runaway costs

Agents fail in characteristic ways, and each maps to a piece of the anatomy. Loops occur when the model repeats an action and observation without progress, burning tokens in circles; this is the most common failure in long sessions. Hallucinated actions occur when the model invents a tool that does not exist, or fabricates a parameter value, or asserts a step succeeded without checking. Error cascades occur when one bad observation contaminates the plan that follows it, and the agent confidently builds on its own mistake.

The cost dimension is easy to underestimate. The chart above shows why: reasoning dominates latency, and by the same token it dominates spend, so a looping agent does not just waste time, it burns metered model calls at scale. Production deployments cap iterations, budget tokens per task, and monitor step-level progress precisely because the failure modes are metered. An agent without a budget is not autonomous; it is unbounded, and the distinction is financial before it is philosophical.

06 Guardrails, sandboxes, and human oversight

The safety layer separates a demo from a deployment. Permission tiers decide which tools the agent may use without asking: read-only access by default, write access gated, irreversible actions such as sending payments or deleting data either prohibited or escalated to a human. Sandboxing constrains where code-execution tools can act, limiting filesystem reach, network access, and runtime so that a hallucinated command damages a container rather than a production system.

Human oversight completes the design. The mature pattern is not supervision of every step, which negates autonomy, but checkpoint approval at consequential boundaries: the plan before execution, the draft before it is sent, the transaction before it commits. Observability, full traces of every prompt, action, and observation, makes post-hoc audit possible, and it is increasingly a procurement requirement. The field's honest summary is that autonomy is granted in scopes, and designing those scopes well is the actual discipline.

07 Where agents are actually deployed today

The adoption curve is steepest where tasks are verifiable, digital, and repetitive: software engineering, where agents write and test code against automatic checks; customer operations, where they draft, classify, and resolve within policy; data work, where they clean, reconcile, and report across systems. Analyst estimates of the agentic market, charted above, show rapid growth from a small base, though such estimates vary widely by firm and should be read as trend indicators rather than measurements.

The deployment map also shows where agents do not yet belong: physical-world control, high-stakes irreversible decisions, and domains with no verification signal. The pattern across both lists supports a sober reading of what agents are: a general architecture for delegating digital work whose reliability is task-dependent, improving generation by generation, and never binary. Understanding the loop, the tools, the memory, and the guardrails, is what lets an organization tell the difference between a task to delegate and one to watch.