Context Engineering for Enterprise AI, Part 3: Multi-Agent Architecture That Survives Production

This is Part 3 of 6 in Context Engineering for Enterprise AI, a series on building a production context-engineering layer for enterprise generative AI. Parts 1 and 2 made each request cheaper and smarter; this part makes the reasoning reliable. We take Mattrx Insights — the agentic product that answers analytical questions with tools — and tear out the single 10k-token mega-agent that loops, stalls, and burns money, replacing it with a supervisor routing to specialized workers under a global budget enforced from the C# boundary.

Recap

• Part 1: Context Management showed how to stop dumping 14,000 tokens into every prompt and compress to ~3,500 engineered tokens, dropping the wrong-answer rate from 18% to 3%.
• Part 2: The Memory Layer added short-term and long-term memory so the system stops re-deriving the same facts every turn.

Both parts were about what goes into one model call. Part 3 is about how many calls there are, who makes them, and when they stop. If you skipped the earlier parts, the short version is: Mattrx is a multi-tenant marketing-analytics SaaS (110k MAU, ~3,200 req/sec peak). The enterprise app, orchestration, API, and governance are ASP.NET Core / .NET 9 on Azure SQL. The AI compute — embeddings, retrieval, agent reasoning, evaluation — is a separate Python FastAPI service using Azure OpenAI + Azure AI Search. The C# app calls Python over HTTP.

TL;DR

Dimension	Mega-agent (before)	Multi-agent supervisor (after)
Topology	1 agent, all tools, 10k-token prompt	1 supervisor + 4 specialized workers
Hand-offs	Free-form text appended to history	Typed contracts + structured state object
Termination	"Stop when you think you're done"	Max-step budget + critic verifier gate
Failure mode	One bad tool call kills the whole run	Per-agent timeout + graceful degradation
p95 latency (agentic query)	4.2s	1.8s
Cost per AI query	$0.021	$0.008
Debuggability	One giant trace, no boundaries	Per-agent spans, isolated retries
C# app API p95	120 ms	120 ms (unchanged)

Production metrics after the migration:

• Agentic query p95 dropped from 4.2s to 1.8s (parallel retrieval + analysis, no runaway loops).
• Cost per AI query fell from $0.021 to $0.008 via model routing (mini for routing/critique, gpt-4o for analysis) plus the context compression from Part 1.
• Wrong-answer / hallucination rate holds at 3% — the critic catches ungrounded claims before they ship.
• Faithfulness eval 0.96, answer-relevance 0.91 — now enforced on agentic output, not just RAG.
• The eval gate blocks any prompt or model change that drops faithfulness below 0.90.
• Runaway loops (>12 model calls) went from ~6% of agentic traffic to 0% — the step budget makes them structurally impossible.
• Per-agent p95: retriever 240 ms, analyst 900 ms, writer 320 ms, critic 280 ms.
• Mattrx Help still deflects ~520 support tickets/month; Insights now answers ~40% of analytical questions without escalation.
• Mid-run cancellations free Azure OpenAI capacity within one step instead of after the full 4.2s run.
• The C# app's own API p95 stays 120 ms — all AI latency lives behind the async Insights endpoint.

The one mental shift

A single agent with every tool and a huge prompt is not "one smart system" — it is a state machine with no states, no guards, and no exits. You do not make it reliable by writing a better prompt. You make it reliable by giving it structure: named roles, typed hand-offs, and a hard stop. The model's job is to decide; your job is to bound what it can decide and when it must stop.

The running example

Mattrx Insights answers questions like "Why did paid-search conversions drop 12% for tenant 4471 last week, and which campaigns drove it?" That requires retrieving metric definitions and recent data, comparing campaigns, writing a grounded explanation, and verifying the explanation follows from the numbers. The first version did all of it inside one Azure OpenAI agent with a 10k-token prompt listing every tool. It worked in demos. In production it looped on ambiguous questions, called the SQL tool eight times in a row, and once spent 4.2 seconds and $0.04 producing a confidently wrong answer. The C# app, which calls the Python service over HTTP, had no way to bound any of it — it just waited.

---

1. Supervisor vs mega-agent

The mega-agent fails because it conflates routing, reasoning, writing, and checking into one context window. Every tool result pollutes the prompt that decides the next tool — by step five the agent is reasoning over its own debris. The fix is a supervisor that classifies the request and routes to specialized workers — retriever, analyst, writer, critic — each with a small prompt and a narrow tool set. The supervisor never touches data; the workers never decide what comes next.

Before

The Python service exposed one agent with everything bolted on. The system prompt alone was ~10k tokens.

# app/agents/mega_agent.py  — the version we deleted
from openai import AzureOpenAI

client = AzureOpenAI(azure_endpoint=settings.aoai_endpoint, api_version="2024-10-21")
MEGA_PROMPT = open("prompts/everything.md").read()  # ~10k tokens, all tools, all rules

async def answer(question: str, tenant_id: int) -> str:
    messages = [{"role": "system", "content": MEGA_PROMPT},
                {"role": "user", "content": question}]
    for _ in range(30):  # "30 should be plenty" — famous last words
        resp = client.chat.completions.create(
            model="gpt-4o", messages=messages, tools=ALL_TOOLS, temperature=0.2)
        msg = resp.choices[0].message
        if not msg.tool_calls:
            return msg.content   # ships whatever it said, grounded or not
        messages.append(msg)
        for call in msg.tool_calls:   # SQL, search, stats, chart... 11 tools, one context
            messages.append({"role": "tool", "tool_call_id": call.id,
                             "content": await dispatch(call, tenant_id)})
    return "I wasn't able to complete that."  # hit 30 steps ~6% of the time

The for _ in range(30) is the whole problem in one line: no budget the model is aware of, no specialization, and a 30-step ceiling that, when hit, both costs the most and returns nothing.

After

A LangGraph supervisor routes to four worker nodes. Each worker is a separate Azure OpenAI call with its own small prompt and only the tools it needs. The supervisor uses a cheap model (gpt-4o-mini) for routing; the analyst uses the capable model.

# app/agents/graph.py
from langgraph.graph import StateGraph, END
from app.agents.state import InsightsState
from app.agents.workers import retriever, analyst, writer, critic
from app.agents.supervisor import route

def build_graph() -> StateGraph:
    g = StateGraph(InsightsState)
    g.add_node("supervisor", route)      # gpt-4o-mini: classify + decide next worker
    g.add_node("retriever", retriever)   # gpt-4o-mini + Azure AI Search
    g.add_node("analyst", analyst)       # gpt-4o + stats/sql tools (the one capable model)
    g.add_node("writer", writer)         # gpt-4o-mini: grounded prose
    g.add_node("critic", critic)         # gpt-4o-mini: verify groundedness
    g.set_entry_point("supervisor")
    # Supervisor returns the next node name (or "done"); workers always report back to it.
    g.add_conditional_edges("supervisor", lambda s: s.next, {
        "retriever": "retriever", "analyst": "analyst",
        "writer": "writer", "critic": "critic", "done": END,
    })
    for worker in ("retriever", "analyst", "writer", "critic"):
        g.add_edge(worker, "supervisor")
    return g.compile()

The supervisor itself is small. It sees a compact state summary (~400 tokens — the question plus what each worker produced so far), not the raw tool debris, and emits a single routing decision on the cheap mini model:

# app/agents/supervisor.py
ROUTER_PROMPT = (
    "Router for an analytics assistant. Pick the single next step: retriever (fetch data), "
    "analyst (compute/compare), writer (draft), critic (verify), done. "
    "Rules: retrieve before analyzing; analyze before writing; always critic before done. "
    'Return JSON: {"next": "...", "reason": "..."}'
)
# The route() function — with its budget + critic guards — is in section 3 below.

Mattrx metric: Splitting one 10k-token agent into a 400-token router plus four narrow workers cut context tokens per agentic request from ~12,000 to ~3,800 and dropped p95 from 4.2s to 2.6s before any parallelism — router decisions are sub-100ms on the mini model.

---

2. Tools and structured hand-offs

The mega-agent passed everything as free-form text in the message history. The analyst read whatever the retriever printed — hedging, apologies, and all. No contract, just hope that the next model could parse the last model's prose. The fix is a typed state object: workers read named fields and write named fields. Hand-offs are data, not narration.

Before

Hand-off by string concatenation — the retriever's output went into the history verbatim and the analyst parsed it by reading English:

# Retriever appended this; the analyst had to extract numbers from prose:
messages.append({"role": "assistant", "content":
    "I found some data. Paid search conversions were around 1,240 last week, "
    "down from about 1,410 the week before, though I'm not sure about campaign 88."})

When the analyst read "around 1,240" as 1240.0 but "about 1,410" as a missing value, the comparison silently used a null. No error — just a wrong answer.

After

A Pydantic state object with explicit fields. Each worker returns a typed delta; nothing is parsed from prose.

# app/agents/state.py
from pydantic import BaseModel, Field

class MetricSeries(BaseModel):
    name: str
    current: float
    prior: float
    tenant_id: int

class InsightsState(BaseModel):
    question: str
    tenant_id: int
    next: str = "supervisor"
    steps: int = 0
    max_steps: int = 8
    retrieved: list[MetricSeries] = Field(default_factory=list)   # retriever writes
    findings: list[str] = Field(default_factory=list)             # analyst writes
    draft: str | None = None                                      # writer writes
    verdict: dict | None = None                                   # critic writes
    trace: list[tuple[str, str]] = Field(default_factory=list)
    # router_view() renders a ~400-token summary (counts, not bodies) for the supervisor.

The analyst now reads state.retrieved — strongly typed floats — and writes state.findings. Tools are declared with strict JSON schemas so Azure OpenAI returns parseable arguments, not English:

# app/agents/workers.py  (analyst excerpt)
from app.agents.state import InsightsState
from app.llm import chat_tools
from app.tools.stats import pct_change   # deterministic Python, not the model's arithmetic

# Strict schema => Azure OpenAI returns parseable args, not English.
ANALYST_TOOLS = [{
    "type": "function",
    "function": {
        "name": "pct_change",
        "description": "Exact percent change between two numbers.",
        "parameters": {
            "type": "object",
            "properties": {"current": {"type": "number"}, "prior": {"type": "number"}},
            "required": ["current", "prior"], "additionalProperties": False,
        },
    },
}]

async def analyst(state: InsightsState) -> InsightsState:
    series = "\n".join(f"{m.name}: current={m.current} prior={m.prior}" for m in state.retrieved)
    result = await chat_tools(
        model="gpt-4o",   # the one capable-model call in the whole graph
        system="Compare the series. Use pct_change for every delta — never compute in prose.",
        user=series, tools=ANALYST_TOOLS, tool_impls={"pct_change": pct_change},
    )
    state.findings = result.findings   # typed list[str], each citing a metric name
    state.trace.append(("analyst", f"{len(result.findings)} findings"))
    return state

The C# side mirrors these contracts so the boundary is type-safe end to end. The orchestration service deserializes the Python result into a record, not a JObject:

// Mattrx.Insights/Contracts/InsightsResult.cs
public sealed record MetricSeries(string Name, double Current, double Prior, int TenantId);

public sealed record InsightsResult(
    string Answer,
    IReadOnlyList<string> Findings,
    Verdict Verdict,
    int Steps,
    decimal CostUsd);

public sealed record Verdict(bool Grounded, double Faithfulness, string? Reason);

Mattrx metric: Typed hand-offs eliminated the silent-null class of bugs — the "around 1,240" parsing failures that corrupted ~2% of analyst comparisons dropped to zero, and the analyst's tool-argument retry rate fell from 9% to under 1% once schemas used additionalProperties: false.

Diagnostic line we use when a hand-off looks wrong:

# Replay one run's state transitions from the trace table
curl -s "$INSIGHTS_API/runs/$RUN_ID/trace" | jq -r '.[] | "\(.[0])\t\(.[1])"'

---

3. Bounded loops and termination

A mega-agent decides when it's done — the single most expensive sentence in agentic systems. The model has no incentive to stop, no awareness of its budget, and no external check on whether its answer is correct. So it loops. The fix has two parts: a hard step budget the supervisor enforces, and a critic agent that verifies the draft against the retrieved data before the run can finish.

Before

The mega-agent's only stop conditions (above) were "the model emitted no tool call" or "we hit 30 iterations." Neither checks correctness: if not msg.tool_calls: return msg.content ships whatever it said, grounded or not. Roughly 6% of runs hit the 30-step ceiling and returned nothing. Of the runs that did terminate, the ungrounded ones shipped straight to the user — the bulk of the 18% wrong-answer rate, and it persisted on agentic queries even after the Part 1 retrieval work landed.

After

The supervisor refuses to route past max_steps, and routes to the critic before it will ever return done. The critic re-scores the draft against the actual retrieved numbers and can bounce it back once.

# app/agents/supervisor.py  (termination guards added)
async def route(state: InsightsState) -> InsightsState:
    if state.steps >= state.max_steps:
        state.next = "writer" if state.draft is None else "done"  # graceful wrap-up
        state.trace.append(("supervisor", "budget exhausted — forcing wrap-up"))
        return state

    decision = await chat_json(model="gpt-4o-mini", system=ROUTER_PROMPT,
                               user=state.router_view())
    nxt = decision["next"]
    # Never finish without a passing critic verdict.
    if nxt == "done" and (state.verdict is None or not state.verdict["grounded"]):
        nxt = "critic"
    state.next = nxt
    state.steps += 1
    state.trace.append(("supervisor", decision["reason"]))
    return state

The critic is a cheap, focused verifier. It does not rewrite — it judges, and it has access only to the draft and the typed numbers:

# app/agents/workers.py  (critic)
CRITIC_PROMPT = (
    "Verify the draft. Every numeric or causal claim must be supported by the provided "
    "metrics. Return JSON: {\"grounded\": bool, \"faithfulness\": 0..1, \"reason\": str}. "
    "If any claim lacks support, grounded=false."
)

async def critic(state: InsightsState) -> InsightsState:
    facts = "\n".join(f"{m.name}: {m.current} vs {m.prior}" for m in state.retrieved)
    verdict = await chat_json(
        model="gpt-4o-mini", system=CRITIC_PROMPT,
        user=f"DRAFT:\n{state.draft}\n\nFACTS:\n{facts}",
    )
    state.verdict = verdict
    if not verdict["grounded"]:
        state.draft = None           # force exactly one rewrite, then budget takes over
    state.trace.append(("critic", f"grounded={verdict['grounded']} f={verdict['faithfulness']}"))
    return state

This is the same eval discipline as the offline gate, run inline: the critic's faithfulness mirrors the offline faithfulness metric, and the 0.90 floor that blocks deploys also blocks an individual answer at runtime.

Mattrx metric: The step budget drove runaway runs (>12 model calls) from ~6% of traffic to 0%, and the inline critic holds the agentic wrong-answer rate at 3% with faithfulness 0.96 — the same number we hold offline. The critic adds one cheap call (~280 ms p95) and pays for itself by killing the $0.04 wrong-answer runs.

---

4. Parallelism and failure isolation

The mega-agent was sequential by construction: one message history, one call at a time. If the SQL tool timed out on step three, the whole run died and the user got nothing after a three-second wait. Independent work — fetching metric definitions and recent data — ran one after the other for no reason. The fix: run independent workers concurrently, give each its own timeout, and degrade gracefully when one fails instead of failing the run.

Before

Sequential retrieval, no per-step timeout, all-or-nothing failure: definitions = await fetch_metric_definitions(tenant_id) (~200 ms) then recent = await fetch_recent_metrics(tenant_id, "7d") (~600 ms, sometimes hangs forever). Two independent fetches run back to back — ~800 ms minimum, and a hang on either with no timeout killed the entire run.

After

The retriever fans out independent fetches with asyncio.gather, each wrapped in its own timeout. A failed sub-fetch degrades to partial data and a flag, not an exception that aborts the graph.

# app/agents/workers.py  (retriever)
import asyncio
from app.agents.state import InsightsState, MetricSeries
from app.search import fetch_definitions, fetch_recent   # Azure AI Search + Azure SQL via API

async def _bounded(coro, timeout: float, label: str):
    try:
        return await asyncio.wait_for(coro, timeout)
    except (asyncio.TimeoutError, Exception):   # degrade, don't crash the run
        return {"_degraded": label}

async def retriever(state: InsightsState) -> InsightsState:
    defs, recent = await asyncio.gather(
        _bounded(fetch_definitions(state.tenant_id), 0.4, "definitions"),
        _bounded(fetch_recent(state.tenant_id, "7d"), 0.9, "recent"),
    )
    if "_degraded" not in recent:
        state.retrieved = [MetricSeries(**r, tenant_id=state.tenant_id) for r in recent]
    else:
        state.findings.append("Note: recent data partially unavailable; answer is best-effort.")
    state.trace.append(("retriever", f"defs={'_degraded' not in defs} recent={'_degraded' not in recent}"))
    return state

The C# orchestration boundary is where the whole run gets its hard ceiling, cost cap, and governance: the Python graph bounds individual steps, C# bounds the request and enforces tenant policy. This is the durable endpoint the enterprise app calls:

// Mattrx.Insights/InsightsOrchestrator.cs
public sealed class InsightsOrchestrator(
    HttpClient http, IGovernance gov, ILogger<InsightsOrchestrator> log)
{
    private static readonly TimeSpan GlobalBudget = TimeSpan.FromSeconds(8);

    public async Task<InsightsResult> AnswerAsync(InsightsRequest req, CancellationToken ct)
    {
        // Governance first: prompt-injection screen + per-tenant cost cap, before any Python call.
        await gov.AssertAllowedAsync(req.TenantId, req.Question, ct);

        using var cts = CancellationTokenSource.CreateLinkedTokenSource(ct);
        cts.CancelAfter(GlobalBudget);   // hard wall-clock cap over the whole Python graph
        try
        {
            using var resp = await http.PostAsJsonAsync("/agentic/answer", req, cts.Token);
            resp.EnsureSuccessStatusCode();
            var result = await resp.Content.ReadFromJsonAsync<InsightsResult>(cts.Token)
                         ?? throw new InvalidOperationException("empty agent result");
            await gov.RecordSpendAsync(req.TenantId, result.CostUsd, ct);
            return result;
        }
        catch (OperationCanceledException) when (cts.IsCancellationRequested && !ct.IsCancellationRequested)
        {
            // The graph blew the global budget — degrade, don't 500 the enterprise app.
            log.LogWarning("Insights global budget exceeded for tenant {Tenant}", req.TenantId);
            return InsightsResult.Degraded("That analysis took too long. Try narrowing the date range.");
        }
    }
}

Registered as a typed HttpClient with a resilience pipeline so a flaky Python instance trips a circuit breaker instead of dragging the C# thread pool down. Agentic calls are too expensive to hammer, so the retry budget is one:

// Program.cs
builder.Services.AddHttpClient<InsightsOrchestrator>(c =>
        c.BaseAddress = new Uri(builder.Configuration["Insights:PythonBaseUrl"]!))
    .AddStandardResilienceHandler(o =>
    {
        o.Retry.MaxRetryAttempts = 1;        // agentic calls are expensive — retry once, max
        o.CircuitBreaker.FailureRatio = 0.3;
    });

Mattrx metric: Parallel retrieval plus per-step timeouts took p95 from 2.6s (post-split, still sequential) to 1.8s, and graceful degradation turned hard failures — Azure SQL spikes, a slow Search shard — into best-effort answers. The Insights endpoint's 500 rate dropped from 1.4% to 0.2%; the rest now return a degraded-but-useful answer. The C# app's own API p95 stayed at 120 ms — all this latency lives behind the async Insights call, never on the core app's request path.

---

Diagrams

Supervisor / worker topology — the supervisor owns control flow, workers own work:

                          ┌──────────────────────────────────────┐
   ASP.NET Core           │           Python AI service           │
   (orchestration,        │             (FastAPI)                 │
    governance,           │                                       │
    cost cap, global      │            ┌────────────┐             │
    timeout)              │     ┌─────▶│ SUPERVISOR │◀─────┐      │
        │   HTTP          │     │      │ gpt-4o-mini│      │      │
        ├────────────────┼─────┤      └─────┬──────┘      │      │
        │                │     │            │ routes      │      │
        │                │   reports     ┌──┴───┬─────┬───┴──┐   │
        ▼                │   back        ▼      ▼     ▼      ▼   │
   ┌─────────┐           │        ┌─────────┐ ┌────────┐ ┌──────┐ ┌────────┐
   │ Verdict │◀──────────┼────────│RETRIEVER│ │ANALYST │ │WRITER│ │ CRITIC │
   │ + answer│  result   │        │ +Search │ │ +stats │ │      │ │ verify │
   └─────────┘           │        │ mini    │ │ gpt-4o │ │ mini │ │  mini  │
                         │        └─────────┘ └────────┘ └──────┘ └────────┘
                         └──────────────────────────────────────────────────┘
   Model routing: capable model (gpt-4o) only for the analyst; everything else on mini.

Single request flowing through the graph, with the budget and the critic gate visible:

 user question
      │
      ▼
 [C# governance: injection screen + tenant cost cap]   ── fail ──▶ 403 / blocked
      │ ok
      ▼
 [C# global budget: 8s linked CancellationToken] ───────────────┐
      │                                                          │ exceeded
      ▼                                                          ▼
 SUPERVISOR ──step 1──▶ RETRIEVER (parallel: defs ‖ recent, per-fetch timeout)
      ▲                     │ writes state.retrieved (typed)
      │◀────────────────────┘
 SUPERVISOR ──step 2──▶ ANALYST (gpt-4o + pct_change tool) ─▶ state.findings
      ▲────────────────────┘
 SUPERVISOR ──step 3──▶ WRITER ─▶ state.draft
      ▲────────────────────┘
 SUPERVISOR ──step 4──▶ CRITIC ─▶ verdict.grounded?
      │                              │
      │           grounded=false ────┘ (draft cleared, ONE rewrite, then budget wins)
      │           grounded=true
      ▼
   "done"  ──▶  C# records spend, returns InsightsResult
   (or, if budget exhausted at any step: forced wrap-up ──▶ degraded answer)

---

Aggregate metrics

Metric	Before (mega-agent)	After (multi-agent)
Agentic query p95 latency	4.2s	1.8s
Cost per AI query	$0.021	$0.008
Context tokens per agentic request	~12,000	~3,800
Wrong-answer / hallucination rate	18% (pre-RAG) → lingering on agentic	3%
Faithfulness eval score	not measured on agentic	0.96
Answer-relevance eval score	not measured on agentic	0.91
Runaway runs (>12 model calls)	~6%	0%
Insights endpoint 500 rate	1.4%	0.2%
Analyst tool-arg retry rate	9%	<1%
C# app API p95	120 ms	120 ms

---

Pre-ship checklist

• Every worker has its own small system prompt and only the tools it needs — no shared mega-prompt.
• Hand-offs are typed (Pydantic on Python, records on C#); nothing is parsed from prose.
• Tool schemas use additionalProperties: false and required so Azure OpenAI returns parseable arguments.
• The supervisor enforces a max_steps budget and cannot route past it.
• The supervisor cannot return done without a passing critic verdict.
• The critic re-scores faithfulness against the typed retrieved data, not the prose draft.
• Independent fetches run with asyncio.gather, each wrapped in its own timeout.
• A failed sub-fetch degrades to a flagged partial answer, never an unhandled exception.
• The C# boundary enforces a global wall-clock budget via a linked CancellationToken.
• The C# boundary screens for prompt injection and applies the per-tenant cost cap before calling Python.
• The HTTP client has a resilience pipeline (one retry max, circuit breaker) — agentic calls are too expensive to hammer.
• Every run writes a per-agent trace so you can replay routing decisions when something looks wrong.

Honest stuff

• Multi-agent is overkill for single-tool tasks. Mattrx Help (straight RAG over docs) does not use the graph — it's one retrieve-then-answer call. A supervisor and four workers for "search and respond" is architecture astronomy. Reach for this only when reasoning fans out across distinct skills.
• More agents means more places to be wrong. The split fixed runaway loops but added a fixed cost: every query now makes 4–6 calls. We came out ahead only because most of them run on the cheap mini model.
• The critic is not free and not infallible. A confidently wrong critic will rubber-stamp a confidently wrong draft. We keep it cheap and narrow so it's hard for it to be cleverly wrong, and still sample 1% of "grounded" answers for human review.
• Step budgets can truncate hard questions. An 8-step ceiling means a genuinely multi-faceted question sometimes gets a partial answer. We chose "fast and honest about limits" over "slow and complete" — a product call, not a universal truth.
• Parallelism hides ordering bugs. A worker that secretly depended on another's output now races. We caught two of these in staging. If your workers aren't truly independent, don't gather them.
• Cheap-model routing occasionally mis-routes. gpt-4o-mini picks the wrong next worker ~1–2% of the time. The "always critic before done" rule makes that survivable — a mis-route costs a step, not a wrong answer.
• Two languages is two deploys and two on-call surfaces. The C#/Python split is right for Mattrx (the AI compute genuinely needs the Python ecosystem), but the SRE now debugs across an HTTP boundary. Split languages for the libraries, not for fashion.
• Don't migrate everything at once. We ran the mega-agent and the graph side by side behind a flag for two weeks, comparing faithfulness and cost on shadow traffic before cutting over.

The closing mental model

The model decides; the graph governs. Bound what it can decide, type what it hands off, and force it to stop — then a fleet of small agents beats one big one every time.

Three enforceable habits:

1. No worker without a budget. Every agent runs under a step ceiling the supervisor owns and a wall-clock ceiling the C# boundary owns. "Stop when done" is not a stop condition.
2. No hand-off without a type. If one agent's output feeds another's input, it crosses a typed contract — MetricSeries, not "around 1,240." Prose is for users, not for agents.
3. No done without a verdict. The critic gate runs inline against the same faithfulness floor (0.90) that gates your deploys. An answer that can't pass the gate doesn't ship.

Continue the series

This is Part 3 of 6 in Context Engineering for Enterprise AI. (you are here) -> Next: Part 4 — Enterprise AI Design — wiring it all together: governance, evals as a deploy gate, cost controls, and the org design that keeps 5 backend engineers and 1 SRE sane. The full series:

1. Part 1: Context Management
2. Part 2: The Memory Layer
3. Part 3: Multi-Agent Architecture
4. Part 4: Enterprise AI Design
5. Part 5: Multi-Tenant Patterns
6. Part 6: AI & Data Governance

Further reading

• Part 1: Context Management — the compression work that makes a 4–6-call graph affordable.
• Part 2: The Memory Layer — how memory feeds the retriever so workers stop re-deriving facts.
• RAG with Azure OpenAI, Azure SQL, and C# — the retrieval foundation the retriever worker builds on.
• LLM patterns in .NET — orchestration, tool-calling, and the C# boundary patterns used here.
• 14 years of enterprise ASP.NET, Part 4: Azure observability and AI — tracing the per-agent spans this architecture produces.

---

Running a multi-agent system that loops, stalls, or ships ungrounded answers? Tell me about your supervisor and your termination conditions — randhir.jassal@gmail.com. I read every one, and I especially want to hear how you bound your agentic runs.