Context Engineering for Enterprise AI, Part 2: The Memory Layer That Makes Agents Useful

This is Part 2 of 6 in Context Engineering for Enterprise AI — the series building a production context-engineering layer for Mattrx, a multi-tenant marketing-analytics SaaS (110k MAU, ~3,200 req/sec peak) split across an ASP.NET Core / .NET 9 app on Azure SQL and a separate Python (FastAPI) AI compute service. Part 1 fixed what goes into a single prompt. This part fixes what survives between prompts. LLMs are stateless functions: every call starts from zero. Users don't experience the world that way — they expect the assistant to remember yesterday's ticket and that their workspace is on the EU residency tier. That gap between "stateless model" and "remembers me" is the memory layer, and getting it wrong leaks one tenant's data into another's prompt.

Recap

In Part 1 — Context Management we stopped dumping everything into the prompt: a context budget, ranked and compressed chunks, per-request tokens down from ~14,000 to ~3,500, wrong-answer rate from 18% to 3%. That work assumed each request was independent. It is not — a partner who asks Mattrx Insights three questions in a row should not have each treated as a cold start. Memory closes that loop, and it has to plug into the same token budget from Part 1, not blow past it.

TL;DR

Memory is not "stuff the chat history into the prompt." It is a tiered system with explicit write policies, scoped retrieval, and governed forgetting. Here's the decision table for what lives where:

Memory tier	What it holds	Store	Lifetime	Goes in prompt?
Working memory	Current turn: question, retrieved chunks, tool results	In-process (request scope)	One request	Always (it is the request)
Short-term	Last N turns of this session, verbatim	Azure SQL (session rows) + Redis cache	Session TTL (24h)	Recent few turns, raw
Long-term episodic	Summaries of past sessions, decisions, stated facts	Azure SQL (system-of-record) + vector index	TTL 90d, salience-gated	Only if retrieved by relevance
Long-term semantic	Durable tenant/user facts ("EU residency", "uses webhooks")	Azure SQL rows + vector index	Until changed/forgotten	Only if retrieved by relevance
Global	Nothing user-specific. Product docs only (that's RAG, not memory)	Azure AI Search	n/a	Via retrieval, never as "memory"

Mattrx production results after building the memory layer (3-week build, 5 backend engineers + 1 SRE):

• Context tokens per request: ~14,000 (history stuffing) -> ~3,500 (tiered, summarized, retrieved) — held across turns, not just within one.
• Cost per AI query: $0.021 -> $0.008 (compression + routing + not re-sending the full transcript every turn).
• Wrong-answer / hallucination rate: 18% -> 3% (memory removes "I already told you that" failures).
• Faithfulness eval: 0.96; answer-relevance: 0.91.
• Cross-session continuity: new capability — Insights resumes yesterday's debugging thread with zero re-explanation.
• Memory retrieval p95: 74 ms; memory write p95 (async, off hot path): 38 ms.
• Long-term memory writes/day: ~2,800 -> ~410 (salience gate).
• Cross-tenant leak incidents in load + red-team testing: 0 (tenant filter is a hard predicate, not a prompt instruction).
• Right-to-be-forgotten SLA: under 5 minutes end to end, including vector index purge.
• Mattrx Help still deflects ~520 tickets/month; the C# app API p95 stays 120 ms, unchanged.

The one mental shift

The model is a stateless function. Memory is an enterprise database with embeddings bolted on — owned, scoped, audited, and forgettable — not a chat transcript you keep pasting back in.

Treat memory as a governed datastore and everything falls into place: tenant isolation is a WHERE clause, forgetting is a DELETE, salience is a column, and "what does the assistant know about me" is a query you can answer in an audit. Treat it as a transcript and you get unbounded token growth, no isolation guarantee, and a GDPR request you cannot fulfill.

The running example

Mattrx's two AI products stress memory differently. Mattrx Help is RAG over docs, mostly stateless per question — but support conversations span turns ("no, I meant the webhook retry"), so it needs short-term memory and the occasional durable fact. Mattrx Insights is the agentic, tool-using system where memory earns its keep: a partner debugging a conversion drop yesterday wants to resume today without re-pasting their setup. Both run on one memory layer — C# (ASP.NET Core + Azure SQL) is the system-of-record and governance boundary; Python (FastAPI + Azure OpenAI embeddings + Azure AI Search) is the semantic memory: summarization, embedding, ranking. The C# app never lets the Python service decide whose memory to read; it passes a tenant scope the Python side cannot widen.

1. Memory tiers: stop stuffing the whole history into the prompt

Before

The first version of Mattrx Insights did what the tutorials do: keep a list of messages and send all of them every turn.

// BEFORE — InsightsController.cs. Every turn re-sends the entire transcript.
[ApiController]
[Route("api/insights")]
public sealed class InsightsController(IChatClient chat) : ControllerBase
{
    // In-memory per-session list. Grows without bound. Lost on restart.
    private static readonly Dictionary<Guid, List<ChatMessage>> _history = new();

    [HttpPost("ask")]
    public async Task<IActionResult> Ask(AskRequest req, CancellationToken ct)
    {
        var turns = _history.GetValueOrDefault(req.SessionId) ?? new();
        turns.Add(new ChatMessage(ChatRole.User, req.Question));

        // The whole conversation, every time. Token count climbs each turn.
        var answer = await chat.CompleteAsync(turns, cancellationToken: ct);

        turns.Add(answer.Message);
        _history[req.SessionId] = turns;
        return Ok(new { answer = answer.Message.Text });
    }
}

By turn 12 of a long Insights session this prompt carried ~14,000 tokens of transcript, most of it irrelevant. The static Dictionary also meant memory died on every deploy and never crossed instances behind the load balancer — and there was no tenant on any of it.

Diagnostic — watch the prompt grow:

# Prompt tokens per turn, from Log Analytics — they climb linearly with the transcript.
$ az monitor log-analytics query -w $LAW_ID --analytics-query \
  "AppTraces | where Message has 'insights.prompt_tokens' \
   | project turn=toint(Properties.turn), tokens=toint(Properties.prompt_tokens)"
# turn 1 -> 1180,  turn 6 -> 7420,  turn 12 -> 14010   <-- mostly stale transcript

After

Split memory into tiers: only the working tier plus a few recent turns go in raw; everything older becomes retrievable long-term memory.

                      MEMORY TIERS  (read on the way in, write on the way out)

  ┌──────────────────────────────────────────────────────────────────────┐
  │ WORKING MEMORY  (in-process, this request only)                        │
  │   question + retrieved doc chunks (Part 1) + tool results + budget     │
  └───────────────┬──────────────────────────────────────────────────────┘
                  │ reads from ↓                            writes to ↑ (async)
  ┌───────────────┴──────────────────────────────────────────────────────┐
  │ SHORT-TERM  (Azure SQL session rows + Redis cache, TTL 24h)           │
  │   last N turns, verbatim. The "we're mid-conversation" tier.          │
  └───────────────┬──────────────────────────────────────────────────────┘
                  │ summarize + salience-gate on close ↓
  ┌───────────────┴──────────────────────────────────────────────────────┐
  │ LONG-TERM  (Azure SQL system-of-record + Azure AI Search vectors)     │
  │   episodic: session summaries, decisions   TTL 90d                    │
  │   semantic: durable tenant/user facts      until changed/forgotten    │
  └────────────────────────────────────────────────────────────────────────┘
                  ↑ retrieved by similarity + recency, merged into Part 1 budget

The C# memory store is the system-of-record. Tier is a column; tenant is non-nullable.

// AFTER — MemoryRecord.cs + EF Core config. Tier-aware, tenant-scoped, governed.
public enum MemoryTier { ShortTerm = 0, Episodic = 1, Semantic = 2 }

public sealed class MemoryRecord
{
    public long Id { get; init; }
    public required Guid TenantId { get; init; }      // hard isolation key
    public required Guid UserId { get; init; }
    public Guid? SessionId { get; init; }             // null for durable semantic facts
    public required MemoryTier Tier { get; init; }
    public required string Content { get; init; }     // verbatim turn / summary / fact
    public double Salience { get; init; }             // 0..1, from the write policy
    public bool ContainsPii { get; init; }
    public DateTimeOffset CreatedAt { get; init; }
    public DateTimeOffset ExpiresAt { get; init; }    // TTL — drives forgetting
}

// DbContext config: never let a query forget the tenant.
modelBuilder.Entity<MemoryRecord>(e =>
{
    e.HasIndex(m => new { m.TenantId, m.UserId, m.SessionId, m.Tier });
    e.HasIndex(m => m.ExpiresAt);                      // TTL sweeper
    e.Property(m => m.TenantId).IsRequired();
    // A forgotten/expired row is invisible to every read path.
    e.HasQueryFilter(m => m.ExpiresAt > DateTimeOffset.UtcNow);
});

Now the prompt carries working memory + the last 3 raw turns + a handful of retrieved long-term memories, all inside the Part 1 budget. The transcript no longer grows linearly.

Mattrx metric: tiering held context tokens per request at ~3,500 even on turn-30 Insights sessions, versus ~14,000 under transcript stuffing — the prompt stopped growing with conversation length.

2. Writing memory: a policy, not "append everything"

Before

The naive version remembered nothing durable; the next version remembered everything — every turn became a long-term row, including "ok thanks." The store filled with noise, retrieval surfaced garbage, and we paid to embed every line.

# BEFORE — memory_writer.py. Persist every turn. No summarization, no dedup, no salience.
async def remember(session_id: str, role: str, text: str) -> None:
    vector = await embed(text)                 # embed even "ok thanks"
    await vector_store.upsert({
        "id": str(uuid4()),
        "session_id": session_id,              # no tenant! no salience! no TTL!
        "text": text,
        "vector": vector,
    })
# Result: 40k near-useless rows in two weeks, retrieval precision in the gutter.

After

A write policy decides what is worth remembering, summarizes old turns into compact episodic memory on session close, dedupes against existing facts, and scores salience so retrieval prefers signal over chatter. Summarization and embedding are Python's job; C# decides whether to call it and owns the durable row.

# AFTER — memory_policy.py (FastAPI service). Summarize, dedup, score salience.
SALIENCE_PROMPT = (
    "Rate 0.0-1.0 how useful this message is to remember long-term for future "
    "analytics questions. Constraints, decisions, account facts, goals score high. "
    "Pleasantries score ~0. Reply with only a number."
)

async def score_salience(text: str) -> float:
    resp = await aoai.chat.completions.create(
        model="gpt-4o-mini",                          # cheap model for the gate
        messages=[{"role": "system", "content": SALIENCE_PROMPT},
                  {"role": "user", "content": text}],
        temperature=0, max_tokens=4)
    try:
        return max(0.0, min(1.0, float(resp.choices[0].message.content.strip())))
    except ValueError:
        return 0.0

async def summarize_session(turns: list[str]) -> str:
    """Collapse a closed session into one episodic memory (~120 tokens)."""
    resp = await aoai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "system", "content": "Summarize into durable facts and "
                   "decisions only. No filler. Max 120 tokens."},
                  {"role": "user", "content": "\n".join(turns)}],
        temperature=0.1, max_tokens=180)
    return resp.choices[0].message.content.strip()

async def is_duplicate(tenant_id: str, user_id: str, text: str) -> bool:
    """If a near-identical fact already exists for this user, skip the write."""
    hits = await semantic_search(tenant_id, user_id, await embed(text), top_k=1)
    return bool(hits) and hits[0].score >= 0.93       # cosine; very-close = dup

The C# side runs this off the hot path: the answer returns immediately and the write happens in a background channel, so persistence never adds latency.

// AFTER — MemoryWriteService.cs. Off-hot-path write with policy + tenant + TTL.
public sealed class MemoryWriteService(
    AppDbContext db,
    IAiMemoryClient ai,                 // typed HttpClient to the Python service
    ILogger<MemoryWriteService> log)
{
    public async Task ConsiderTurnAsync(
        TenantScope scope, string userText, CancellationToken ct)
    {
        var salience = await ai.ScoreSalienceAsync(userText, ct);
        if (salience < 0.55) return;                          // not worth remembering
        if (await ai.IsDuplicateAsync(scope, userText, ct)) return;

        db.Memories.Add(new MemoryRecord
        {
            TenantId    = scope.TenantId,                     // never from client input
            UserId      = scope.UserId,
            SessionId   = scope.SessionId,
            Tier        = MemoryTier.Semantic,
            Content     = userText,
            Salience    = salience,
            ContainsPii = PiiScanner.Detect(userText),
            CreatedAt   = DateTimeOffset.UtcNow,
            ExpiresAt   = DateTimeOffset.UtcNow.AddDays(90),
        });
        await db.SaveChangesAsync(ct);
        log.LogInformation("memory.write tenant={T} salience={S:0.00}",
            scope.TenantId, salience);
    }
}

Diagnostic — the salience gate's effect on write volume:

$ az monitor log-analytics query -w $LAW_ID --analytics-query \
  "AppTraces | where Message startswith 'memory.write' | summarize count() by bin(TimeGenerated,1d)"
# Before policy: ~2,800 writes/day.  After: ~410/day.  85% of turns weren't worth remembering.

Mattrx metric: the write policy cut long-term rows ~85% (2,800 -> ~410 writes/day) while raising retrieval precision — the store stopped being full of "thanks, that helped."

3. Retrieving memory: relevance + recency + tenant scope, inside the Part 1 budget

Before

The first "use memory" attempt re-sent the full short-term history and ignored long-term entirely — same token-growth problem as Section 1, and it never recalled anything from a previous session. There was no notion of relevant memory; it was all-or-nothing.

# BEFORE — retrieve.py. Returns the whole session blob. No ranking, no recency, no tenant.
async def get_memory(session_id: str) -> str:
    rows = await db.fetch("SELECT text FROM turns WHERE session_id=$1", session_id)
    return "\n".join(r["text"] for r in rows)         # everything, unranked

After

Memory retrieval is its own ranked query: embed the question, fetch candidates filtered by tenant and user, re-rank by a blend of similarity and recency. The result merges into the same Part 1 budget — short-term raw turns first, then top long-term memories until the budget is spent.

              PER-REQUEST MEMORY RETRIEVAL FLOW

   C# InsightsController                Python AI service (FastAPI)
   ─────────────────────                ──────────────────────────
   request + TenantScope ───HTTP──▶  POST /memory/retrieve
                                       │ 1. embed(question)        ~18ms
                                       │ 2. vector search WHERE     ~31ms
                                       │      tenant_id = scope AND
                                       │      user_id  = scope
                                       │ 3. score = 0.7*sim         ~2ms
                                       │            + 0.3*recency
                                       │ 4. take top-k under budget
        ranked memories  ◀────────────┘ p95 total ~74ms
        │
        ▼
   merge into Part 1 budget:
   [ system ] [ recent 3 raw turns ] [ top long-term memories ] [ retrieved docs ]
                       (short-term)         (episodic+semantic)        (RAG, Part 1)
        │
        ▼  single ~3,500-token prompt  ──▶  model

The ranking blend lives in Python, next to the embeddings:

# AFTER — memory_retrieval.py. Tenant-scoped vector search + recency blend.
from datetime import datetime, timezone

async def retrieve_memories(
    tenant_id: str, user_id: str, question: str,
    token_budget: int, half_life_days: float = 14.0,
) -> list[dict]:
    qvec = await embed(question)

    # Tenant + user are HARD filters in the vector query — not prompt instructions.
    candidates = await search_client.search(
        search_text=None,
        vector_queries=[VectorizedQuery(vector=qvec, k_nearest_neighbors=30,
                                        fields="content_vector")],
        filter=f"tenant_id eq '{tenant_id}' and user_id eq '{user_id}'",
        top=30,
    )

    now = datetime.now(timezone.utc)
    scored = []
    for c in candidates:
        age_days = (now - c["created_at"]).total_seconds() / 86400.0
        recency = 0.5 ** (age_days / half_life_days)        # exponential decay
        blended = 0.7 * c["@search.score"] + 0.3 * recency
        scored.append({**c, "blended": blended})

    scored.sort(key=lambda x: x["blended"], reverse=True)

    # Pack into the Part 1 budget: stop when we'd overflow.
    picked, used = [], 0
    for m in scored:
        cost = approx_tokens(m["content"])
        if used + cost > token_budget:
            break
        picked.append(m); used += cost
    return picked

And the C# caller — the TenantScope is derived server-side from the authenticated session, never the request body, so the Python service can only ever search within the caller's tenant:

// AFTER — typed client call. Tenant scope is server-derived and non-overridable.
public async Task<IReadOnlyList<Memory>> RetrieveAsync(
    TenantScope scope, string question, int tokenBudget, CancellationToken ct)
{
    var payload = new { tenant_id = scope.TenantId, user_id = scope.UserId, // from auth
                        question, token_budget = tokenBudget };             // Part 1 leftover
    using var resp = await _http.PostAsJsonAsync("/memory/retrieve", payload, ct);
    resp.EnsureSuccessStatusCode();
    return await resp.Content.ReadFromJsonAsync<List<Memory>>(cancellationToken: ct) ?? [];
}

Diagnostic — confirm retrieval respects the budget and tenant. Posting a question for t_42/u_9 and inspecting the result with jq '{count: length, tenants: (map(.tenant_id)|unique)}' returns { "count": 4, "tenants": ["t_42"] } — four memories, ~870 tokens, one tenant.

Mattrx metric: scoped, ranked retrieval made cross-session continuity real — Insights resumes yesterday's thread with the 4 most relevant memories (~870 tokens) instead of 30 raw turns, holding the request at ~3,500 tokens and cost at $0.008.

4. Governance, isolation, and forgetting

Before

The early store had a single shared namespace. One tenant's facts could surface in another's prompt the moment embeddings were close enough — two customers both debugging "Shopify webhook retries" pulled each other's account specifics. And there was no way to honor a deletion request: memory lived in three places (SQL, vector index, Redis) and nobody could purge all three.

# BEFORE — the leak. Search ignores tenant; nearest neighbors cross customers.
hits = await search_client.search(
    vector_queries=[VectorizedQuery(vector=qvec, k_nearest_neighbors=10,
                                    fields="content_vector")],
    # no filter -> "Shopify webhook" facts from tenant A surface for tenant B
)

After

Isolation is a hard predicate on every read (Section 3's filter=), and the C# system-of-record owns three governance guarantees: TTL forgetting, PII handling, and right-to-be-forgotten that fans out to all three stores in one operation.

// AFTER — ForgetService.cs. Right-to-be-forgotten across SQL + vectors + cache.
public sealed class ForgetService(
    AppDbContext db, IAiMemoryClient ai, IDistributedCache cache,
    IAuditLog audit, ILogger<ForgetService> log)
{
    /// Hard-deletes all memory for a user across every store. GDPR Art. 17.
    public async Task ForgetUserAsync(
        TenantScope scope, Guid targetUserId, CancellationToken ct)
    {
        // 1. System-of-record: physical delete (not the TTL soft-hide).
        var rows = await db.Memories
            .Where(m => m.TenantId == scope.TenantId && m.UserId == targetUserId)
            .ExecuteDeleteAsync(ct);

        // 2. Vector index: purge embeddings for the same scope.
        await ai.PurgeVectorsAsync(scope.TenantId, targetUserId, ct);

        // 3. Cache: evict short-term session blobs.
        await cache.RemoveAsync($"mem:{scope.TenantId}:{targetUserId}", ct);

        // 4. Audit: who asked, what was purged, when. Immutable record.
        await audit.WriteAsync(new AuditEntry(
            Actor: scope.UserId, Action: "memory.forget",
            Subject: targetUserId, Tenant: scope.TenantId,
            Detail: $"rows={rows}", At: DateTimeOffset.UtcNow), ct);

        log.LogWarning("memory.forget tenant={T} user={U} rows={R}",
            scope.TenantId, targetUserId, rows);
    }
}

The vector purge on the Python side mirrors the same hard tenant + user predicate:

# AFTER — purge.py. Forgetting fans out to the vector store too.
async def purge_vectors(tenant_id: str, user_id: str) -> int:
    to_delete = await search_client.search(
        search_text="*",
        filter=f"tenant_id eq '{tenant_id}' and user_id eq '{user_id}'",
        select="id", top=1000,
    )
    keys = [{"id": d["id"]} for d in to_delete]
    if keys:
        await search_client.delete_documents(documents=keys)
    return len(keys)

TTL forgetting runs as a scheduled sweep, and the EF HasQueryFilter from Section 1 makes expired rows invisible to reads before the physical sweep runs — forgetting is correct even between sweeps.

Diagnostic — prove a forgotten user leaves nothing behind. After ForgetUserAsync for u_9 in t_42, sqlcmd -Q "SELECT COUNT(*) FROM Memories WHERE TenantId='t_42' AND UserId='u_9'" returns 0, and the same /memory/retrieve call piped through jq 'length' also returns 0 — SQL, vectors, and cache all clear in one round-trip.

Mattrx metric: with tenant + user as hard filters and a fan-out forget path, cross-tenant leak incidents in load and red-team testing held at 0, and right-to-be-forgotten completes in under 5 minutes end to end, vector purge included.

Aggregate metrics

Metric	Before (history stuffing, no governed memory)	After (tiered, governed memory)
Context tokens / request (turn 30)	~14,000	~3,500
Cost / AI query	$0.021	$0.008
Wrong-answer / hallucination rate	18%	3%
Faithfulness eval	0.71	0.96
Answer-relevance eval	0.68	0.91
Cross-session continuity	none	resumes prior sessions
Long-term memory writes / day	~2,800 (everything)	~410 (salience-gated)
Memory retrieval p95	n/a (re-sent all)	74 ms
Memory write p95 (off hot path)	n/a	38 ms
Cross-tenant leak incidents (testing)	observed	0
Right-to-be-forgotten SLA	impossible (3 stores, no path)	< 5 min
C# app API p95	120 ms	120 ms (unchanged)

Pre-ship checklist

• Every memory row has a non-nullable TenantId; the column is in the primary read index.
• Tenant and user are hard filters in the vector query, never prompt instructions ("only use tenant X" is not isolation).
• TenantScope is derived from the authenticated session server-side and cannot be set from the request body.
• Short-term memory has a TTL (24h) and falls back to SQL when the Redis cache misses.
• A write policy gates long-term writes on salience; pleasantries do not persist.
• Old sessions are summarized into one episodic memory on close; raw transcripts are not kept long-term.
• Dedup runs before every semantic write (cosine >= 0.93 means skip).
• Memory retrieval blends similarity + recency and packs into the leftover Part 1 token budget, not on top of it.
• PII is detected on write and flagged; PII-bearing memory has a shorter TTL.
• Right-to-be-forgotten fans out to SQL + vector index + cache in one operation and writes an audit entry.
• An EF global query filter hides expired rows even before the TTL sweep runs.
• Eval gate re-runs faithfulness on a memory-using sample and blocks any change that drops it below 0.90.

Honest stuff

• Memory is a liability surface, not just a feature. Every durable fact must be isolated, secured, and deletable. If your product doesn't need cross-session recall, don't build long-term memory — short-term session state is cheaper to govern.
• LLM salience scoring costs money and can be wrong. Our gpt-4o-mini gate occasionally rates a useful constraint low. We accept the miss because it degrades gracefully (the user restates it); a wrongly-remembered PII fact does not.
• Summarization loses detail. Collapsing a session to 120 tokens drops nuance — fine for debugging threads, wrong for anything legal or financial. Keep verbatim turns there and pay the tokens.
• Recency decay is a heuristic. A 14-day half-life fits analytics questions but not slow-moving account facts. We pin true semantic facts to no decay and only decay episodic memories.
• Three stores must stay consistent. Forgetting must hit SQL, vectors, and cache; a partial delete is a compliance failure that looks like success. Test the fan-out, not just the SQL delete.
• Per-tenant vector filtering costs at scale. Filtered search is slower than unfiltered; at 110k MAU we keep it under 74 ms p95, but a tenant explosion may force per-tenant indexes.
• "Remembering" can creep users out. Recalling too much, too eagerly reads as surveillance. We surface memory only when retrieved as relevant, never "as you mentioned three weeks ago…" unprompted.
• Don't put global product knowledge in memory. Docs belong in RAG (Part 1). Mixing the two is how a doc edit fails to propagate and a stale "fact" haunts every prompt.
• Don't build memory before your evals can measure it. If you can't tell whether memory improved faithfulness, you can't tell whether a bug regressed it. Wire the eval gate first.

The closing mental model

Memory is a governed database with embeddings — scoped by a WHERE clause, forgotten by a DELETE, and trusted only because both are audited.

Three enforceable habits:

1. Tenant + user are predicates, not prompt text. If isolation depends on the model obeying an instruction, it isn't isolation. Enforce it in the query.
2. Write through a policy, retrieve into a budget. Nothing persists without salience; nothing is recalled without earning its tokens against the Part 1 budget.
3. If you can store it, you must be able to forget it. Build the right-to-be-forgotten path on day one and test that it clears every store.

Continue the series

This is Part 2 of 6 in Context Engineering for Enterprise AI. (You are here.)

-> Next: Part 3 — Multi-Agent Architecture — splitting one overloaded mega-agent into scoped, coordinated agents so the agentic query p95 drops from 4.2s to 1.8s.

The full series:

1. Part 1: Context Management
2. Part 2: The Memory Layer (you are here)
3. Part 3: Multi-Agent Architecture
4. Part 4: Enterprise AI Design
5. Part 5: Multi-Tenant Patterns
6. Part 6: AI & Data Governance

Further reading

• Part 1: Context Management — the token budget that this part's memory retrieval plugs into.
• Part 3: Multi-Agent Architecture — how scoped agents each get their own slice of the memory layer.
• RAG with Azure OpenAI, Azure AI Search and C# — the retrieval foundation Mattrx Help is built on.
• LLM patterns in .NET — the Semantic Kernel / HttpClient patterns used in the C# code here.
• Azure observability and AI for enterprise ASP.NET — the Log Analytics diagnostics used to measure memory writes and token growth.

---

Building tenant-scoped memory for an enterprise assistant and unsure where the isolation boundary should sit, or how to make right-to-be-forgotten actually clear every store? Email me at randhir.jassal@gmail.com with how your memory layer is structured and I'll send back the failure modes I'd red-team first.