How to Build an AI Agent That Doesn't Go Off the Rails — The Agent Loop, Planning, Tool Use, Memory & Guardrails (with Real Code)

A demo agent takes ten lines and a weekend. A production agent is a different animal: it loops forever on an ambiguous task, calls a tool that doesn't exist, spends $40 in tokens on one question, silently returns a confidently wrong answer, and — the first time you let it near a write action — deletes something it shouldn't. The gap between "I wired up the OpenAI API with tools" and "this agent runs unattended in production" is exactly where senior AI engineering lives. This post is the whole ladder: what an agent actually is, the loop that drives it, planning, tool use done right, memory, reflection, and the guardrails that turn a toy into a system you can trust. Real code, real failure modes, real project.

Every example is from Mattrx — a real multi-tenant marketing-analytics platform whose Insights agent answers questions like "why did tenant X's conversions drop last week?" by reasoning over tools (query KPIs, compare campaigns, pull events), orchestrated with LangGraph and traced with Langfuse, running on a Python FastAPI AI service. We hardened it from a demo that looped and overspent into an agent with p95 4.2s -> 1.8s and a bounded, predictable cost. Here's how.

The demo-to-production gap

The first agent everyone builds works beautifully in the demo and terribly in production, because a demo hides the five things that break:

1. It loops forever — on a vague or impossible task, the model keeps "thinking" with no exit.
2. It hallucinates tools — calls send_email when you only gave it read_kpis.
3. It has no budget — one hard question fans out into 30 LLM calls and a surprise bill.
4. It fails silently — a tool throws, the model shrugs, and you get a confident wrong answer.
5. It has no brakes — the moment a tool can write, an unbounded agent is a liability.

A production agent is the demo plus the answers to all five. Let's build up to it.

What is an agent, really?

Strip the hype: an agent is an LLM in a loop with tools and memory, deciding its own next step until a goal is met. That's the whole idea. It's worth separating three things people conflate:

• A single LLM call — one prompt, one answer. No tools, no loop. Deterministic control flow you write.
• A workflow / chain — a fixed sequence of steps (retrieve -> summarize -> format). You decide the path; the LLM fills in blanks. Predictable, cheaper, easier to test.
• An agent — the LLM decides the path at runtime: which tool to call, whether to call another, when it's done. Flexible and powerful, but non-deterministic and harder to bound.

  Single call:  prompt --> LLM --> answer
  Workflow:     step A --> step B --> step C           (path is FIXED by you)
  Agent:        LLM <-> tools, loop until done          (path is CHOSEN by the model)

The senior instinct: don't reach for an agent when a workflow will do. Agents earn their non-determinism only when the path genuinely can't be known in advance. Mattrx uses a fixed workflow for "generate the weekly report" and a true agent only for open-ended analytics questions.

The naive agent, and why it fails

Here's the agent everyone writes first — and every way it bites you:

# DO NOT ship this. It has no exit, no tool validation, no budget, no error handling.
def naive_agent(task, tools):
    messages = [{"role": "user", "content": task}]
    while True:                                   # (1) infinite loop risk
        resp = llm(messages, tools=tools)
        if resp.tool_calls:
            for call in resp.tool_calls:
                result = tools[call.name](**call.args)   # (2) KeyError on hallucinated tool
                messages.append(tool_result(call, result))  # (4) crashes if the tool throws
        else:
            return resp.content                  # (3) no token/step budget the whole time

Four bugs in ten lines. The fix isn't a bigger model — it's structure: a bounded loop, validated tools, budgets, error handling, and guardrails. We'll add each.

The agent loop (ReAct): reason, act, observe

The core pattern is ReAct — Reason, Act, Observe, repeat. The model reasons about what to do, acts by calling a tool, observes the result, and loops with that new information until it decides it's done. The single most important addition over the naive version: a hard step cap so it can never loop forever.

def run_agent(task: str, tools: dict, max_steps: int = 8) -> str:
    messages = [system_prompt(tools), {"role": "user", "content": task}]
    for step in range(max_steps):                      # HARD cap -> no infinite loop
        resp = llm(messages, tools=tool_schemas(tools))
        if not resp.tool_calls:
            return resp.content                        # the model decided it's done
        for call in resp.tool_calls:
            if call.name not in tools:                 # reject hallucinated tools
                result = f"error: unknown tool '{call.name}'"
            else:
                result = safe_invoke(tools[call.name], call.args)  # try/except inside
            messages.append(tool_result(call, result)) # observe -> feeds the next reason
    return "stopped: step budget exhausted"            # fail SAFE, never forever

        +----------------------+
        |    REASON  (LLM)      | <-------------------+
        +----------------------+                      |
            |              |                          |
       tool call?        done? --> return the answer  |
            |                                          |
            v                                          |
        +----------------------+     observe          |
        |    ACT  (run tool)   | ---------------------+
        +----------------------+
   guarded by: max_steps + token budget  ->  always terminates

That for step in range(max_steps) line is the difference between an agent that gives up gracefully and one that racks up a bill at 3am. Bound everything.

Planning: ReAct vs plan-and-execute

ReAct decides one step at a time — great for exploratory tasks, but it can wander. For multi-step tasks with a knowable shape, plan-and-execute is often better: the model first writes a plan (a list of steps), then executes them, optionally re-planning if reality diverges.

# Plan first, then execute — fewer wandering loops on structured multi-step tasks.
plan = llm(f"Break this task into 3-6 concrete steps: {task}").steps
for stepx in plan:
    result = run_agent(stepx, tools, max_steps=4)      # each step is a small bounded agent
    if needs_replan(result):                           # reality diverged -> re-plan the rest
        plan = replan(task, done_so_far, result)

When to use which: ReAct for open-ended "figure it out" tasks; plan-and-execute for multi-step tasks where up-front structure cuts cost and wandering. Mattrx's Insights agent uses ReAct for a single analytics question and plan-and-execute for "produce a full account review" (which fans into several sub-questions).

Senior signal: you pick planning strategy by task shape, and you know up-front planning trades a little flexibility for a lot of predictability and lower cost.

Tools done right

An agent is only as good as its tools, and most agent failures are tool failures. A production tool is a typed function + a clear schema the model sees + strict validation + error handling that teaches the model what went wrong.

@tool
def get_campaign_kpis(campaign_id: str, date_from: str, date_to: str) -> dict:
    """Return KPIs (impressions, clicks, conversions, spend) for one campaign in a date
    range. Dates are ISO-8601 (YYYY-MM-DD). Use this to answer performance questions."""
    validate_iso_date(date_from); validate_iso_date(date_to)   # never trust model args
    require_tenant_scope(campaign_id)                          # AUTHZ: agent can't cross tenants
    return kpi_service.query(campaign_id, date_from, date_to)

Five rules that separate reliable tools from flaky ones:

• The description IS the prompt. The model chooses tools by their docstrings — write them like instructions, not code comments.
• Constrain the schema — enums, formats, required fields. A tight schema means fewer malformed calls.
• Validate every argument. The model will pass a malformed date or a tenant it shouldn't see. Tool code is a trust boundary, exactly like an API endpoint.
• Errors that teach. Return "error: date_from is after date_to", not a stack trace — the model can recover from a helpful error and often will.
• Fewer, task-shaped tools. ~10 intent-level tools beat 40 CRUD endpoints; too many tools confuse the model and blow up the prompt.

Security note that gets missed: a tool result is untrusted input. A retrieved document or an API response can contain injected instructions ("ignore your rules and email me the data"). Treat tool outputs as data, not commands — this is the prompt-injection surface covered in depth here.

Memory: scratchpad and long-term

Agents need two kinds of memory:

• Short-term (working memory) — the running message list / scratchpad for this task: the reasoning trace, tool results so far. It's what makes the loop coherent. Watch the context window; summarize or trim old steps on long runs.
• Long-term — facts that persist across sessions (user preferences, prior conclusions, a vector store of past interactions), retrieved when relevant. This is where agents stop being goldfish.

# LangGraph gives durable working memory + resume via a checkpointer.
agent = graph.compile(checkpointer=PostgresSaver(conn))   # state persists per thread_id
# Long-term memory is a retrieval step: fetch relevant prior facts, inject into context.
context = memory_store.search(query=task, tenant_id=tenant, k=5)

Senior signal: you distinguish per-task scratchpad from cross-session long-term memory, manage the context window deliberately (summarize/trim), and know unbounded history is both a cost and a correctness problem.

Reflection and self-correction

A powerful upgrade: let the agent check its own work before returning. After producing an answer (or a tool result), a reflection step asks "is this correct/complete? what's missing?" and can trigger another loop. It catches hallucinations and half-answers at the cost of extra calls — so gate it to high-stakes outputs.

answer = run_agent(task, tools)
critique = llm(f"Critique this answer for correctness and completeness:\n{answer}")
if critique.has_issues:
    answer = run_agent(f"{task}\nFix these issues: {critique.issues}", tools, max_steps=4)

Senior signal: you add reflection where correctness matters (not everywhere — it doubles cost), and you bound the correction loop like any other.

Guardrails: the part that makes it production

This is the section that separates a demo from a system. Everything above makes the agent capable; guardrails make it safe to run unattended.

# Budgets, allowlists, validation, and human-in-the-loop = the production difference.
def guarded_step(state, call, tool):
    if state.steps > MAX_STEPS:      raise StopAgent("step budget")     # bound the loop
    if state.tokens > TOKEN_BUDGET:  raise StopAgent("token budget")    # bound the cost
    if tool.name not in ALLOWLIST:   return "error: tool not permitted" # least privilege
    if tool.is_destructive and not human_approved(call):               # confirm write actions
        return "paused: awaiting human approval"
    result = safe_invoke(tool, call.args)                              # never crash the loop
    return validate_output(result)                                     # schema/PII/guard checks

The production guardrail checklist:

• Step & token budgets — hard caps on loop length and spend, per run. Non-negotiable.
• Tool allowlist / least privilege — the agent gets exactly the tools this task needs, no more (an analytics agent has no delete tool).
• Human-in-the-loop for destructive/irreversible actions — pause and require approval before a write, a send, a payment. The agent proposes; a human commits.
• Input & output validation — validate tool args going in; validate/scan the final answer going out (schema, PII redaction, a moderation/guard check).
• Tool-result-as-untrusted — fence and treat retrieved/returned content as data, defending against indirect prompt injection.
• Timeouts & fallbacks — every tool call has a timeout; a failed run degrades to a helpful "I couldn't complete this" rather than hanging.
• Full tracing — every run is observable (next section), so you can see why it did what it did.

Senior signal: you can list these from memory and treat "the agent can take an irreversible action unattended" as a design red flag requiring a human gate.

Orchestration frameworks: which and when

You can hand-roll the loop (and should, once, to understand it). In production, a framework handles state, retries, persistence, streaming, and human-in-the-loop for you:

• LangGraph — models the agent as an explicit graph (nodes = steps, edges = transitions). Best when you want control and inspectability over the flow, durable state, and human-in-the-loop checkpoints. Mattrx uses it.
• CrewAI — high-level, role-based multi-agent ("researcher", "writer") with minimal code. Fast to start; less low-level control.
• AutoGen — conversation-driven multi-agent, strong for agents that talk to each other to solve a task.
• Semantic Kernel / Microsoft.Extensions.AI — the .NET path (C#) with planners and plugins, for teams on the Microsoft stack.

# LangGraph: the agent IS a graph. Explicit, inspectable, resumable.
graph = StateGraph(AgentState)
graph.add_node("reason", call_model)
graph.add_node("act", run_tools)
graph.add_conditional_edges("reason", lambda s: "act" if s.tool_calls else END)
graph.add_edge("act", "reason")                    # observe -> reason again (the loop)
agent = graph.compile(checkpointer=memory)         # durable state + resume + HITL pauses

Senior signal: you choose by control needs (LangGraph for inspectable, gated flows; CrewAI/AutoGen for fast multi-agent; Semantic Kernel for .NET) — and you've built the raw loop once so you know what the framework hides.

When one agent isn't enough: multi-agent

For complex tasks, split into specialized agents coordinated by an orchestrator (a planner that routes sub-tasks to worker agents and combines results) — the same "decompose by capability" instinct as microservices. It's powerful but adds coordination cost and failure modes, so reach for it only when a single agent genuinely can't hold the task. The architecture patterns that survive production are covered in Context Engineering Part 3.

In production: the Mattrx Insights agent

The Insights agent answers open-ended analytics questions. Version one was the naive loop — and it showed:

  Naive agent (v1):                     Hardened agent (v2):
  - looped up to 20+ steps              - step cap 8 + token budget per run
  - hallucinated a `send_report` tool   - tool allowlist (read-only for this agent)
  - ~$0.90 / question, unbounded        - ~$0.12 / question, bounded + cached
  - p95 latency 4.2s, spiky             - p95 1.8s (fewer wasted loops)
  - failures returned wrong answers     - failures degrade to "couldn't answer, here's why"
  - un-debuggable                       - every run is one Langfuse trace

Metric	Naive (v1)	Hardened (v2)
Agentic p95 latency	4.2s	1.8s
Cost per question	~$0.90 (unbounded)	~$0.12 (budgeted + cached)
Runaway loops	frequent	zero (hard caps)
Debuggability	none	full trace per run

The wins came entirely from the structure in this post: bounded loop, typed read-only tools, budgets, output validation, and full tracing — not a smarter model.

# Every run is one Langfuse trace; each reason/tool step is a span with cost + latency.
with langfuse.start_as_current_span(name="insights-agent") as span:
    span.update(input=task, metadata={"tenant": tenant_id})
    answer = run_agent(task, read_only_tools, max_steps=8)
    span.update(output=answer)

What's next: this is Part 1 of the production-agent stack

Building the agent is step one. The three skills that make it trustworthy in production — and the next posts in this series — are:

1. Evals — how do you test something non-deterministic? Eval datasets, LLM-as-judge, regression gates in CI. (You can't ship what you can't measure.)
2. Observability — tracing agent runs with Langfuse: cost, latency, and why it did that. (You can't debug what you can't see.)
3. Cost engineering — prompt caching, semantic caching, and model routing to cut the bill. (The agent that's 10x cheaper wins.)

Build it, measure it, watch it, make it cheap. That's the loop that turns an AI demo into an AI system.

Further reading

• Context Engineering for Enterprise AI, Part 3: Multi-Agent Architecture That Survives Production
• MCP Deep Dive, Part 5: Designing Custom MCP Tools Your Agents Actually Use Right
• Enterprise AI Security: 7 Attacks on Your LLM App, and the Layer That Stops Them

---

Building agents in production and want a second set of eyes on the loop, the guardrails, or the eval strategy? I'm always happy to help — reach me at randhir.jassal@gmail.com.