14 Years of Enterprise ASP.NET, Part 4: Azure, Observability & AI in Real Systems

Part 4 of 4 — 14 Years of Enterprise ASP.NET. The finale: where the system actually runs. Three lessons that separate "it works on my machine" from "it runs reliably for 110k users" — choosing Azure architecture by cost and scaling profile, making the system observable so you can fix what you can't see, and treating AI as a real architectural component, not a demo. Real before/after, diagrams, and the Mattrx numbers.

---

Recap

The last part of a four-part series distilling 14+ years of enterprise ASP.NET. Part 1 was code craft, Part 2 the data layer, Part 3 performance and architecture. Now: cloud, observability, and AI.

Running example: Mattrx — .NET 9 / ASP.NET Core, 110k MAU, Azure SQL, ~3,200 req/sec peak, on Azure.

---

Lesson 10 — Azure Cloud Architecture

The cloud lesson that took me longest to learn: pick the compute by your scaling and operational profile, then right-size — don't default to the biggest box or the trendiest platform. Most enterprise .NET runs perfectly on Azure App Service; you reach for Container Apps or AKS when you have a specific reason, not because Kubernetes is on your résumé.

Match the platform to the workload, lean on managed services, and size to real load with autoscale — not to peak-times-three "just in case." Over-provisioning is the most common and most invisible cloud waste; it never pages anyone, so nobody fixes it.

Before — over-provisioned, manually scaled, self-managed

BEFORE — defaulted to "big and always on"
- Web tier: P2v3 × 6, always on (sized for month-end peak, idle the other 27 days)
- Self-hosted Redis on a VM the team patched
- One giant DB tier sized for the worst query
- scaled by hand before known busy periods → over-provisioned the rest of the month

After — right platform, managed services, autoscale

AFTER — sized to real load, elastic, managed
┌──────────────────────────────────────────────────────────┐
│ Azure Front Door / CDN  (edge cache absorbs read traffic)  │
└───────────────┬──────────────────────────────────────────┘
                ▼
   App Service P1v3 × 2  ──(autoscale on CPU + HTTP queue)──► up to 8
                │                          burst worker pool B2 × 0–40 (Reports)
                ▼
   Azure SQL (right-sized + read scale-out)   Managed Redis (HybridCache L2)
- App Service for the web tier (no K8s ops burden for a 5-person team)
- managed Redis instead of a VM the team babysits
- autoscale to peak, scale IN the other 27 days

The decision framework I use: App Service for standard web/API workloads (default), Container Apps when you want containers + scale-to-zero without running a cluster, AKS only when you genuinely need its control plane and have the ops capacity. Mattrx is App Service — a 5-person team has no business running Kubernetes.

Mattrx metric: right-sizing the web tier (P2v3×6 → P1v3×2 + autoscale), moving to managed Redis, and tuning the SQL tier saved roughly $1,300/month (compute) + $280/month (SQL) + $420/month (memory/perf work) — about $2,000/month total — with better peak headroom than the always-on over-provisioned setup, because autoscale handles the month-end burst the fixed fleet was sized for.

---

Lesson 11 — Observability is Essential

For years I "had logging" and was still blind in production. The shift from logging to observability — being able to answer new questions about a running system without shipping new code — is the difference between a 4-minute incident and a 4-hour one. The three pillars: structured logs, metrics (rates, latencies, saturation), and traces (one request across components), tied together by a correlation ID.

You can't fix what you can't see, and you can't see what you didn't instrument. Logs tell you what happened, metrics tell you how much/how often, traces tell you where the time went. Without all three you're guessing — and guessing in production is expensive.

Before — logs as printf, no correlation

// BEFORE — unstructured strings, no trace, no way to find ONE user's request
_logger.LogInformation("Getting campaigns for " + tenantId);   // string-concat, unsearchable
// when it breaks at 3:42pm you grep by timestamp and hope

After — structured logs + OpenTelemetry traces + metrics

// AFTER — structured fields + a correlation scope so every line in the request is linkable
using (logger.BeginScope(new Dictionary<string, object> { ["CorrelationId"] = correlationId }))
{
    logger.LogInformation("Fetched {Count} campaigns for {TenantId} in {Ms}ms",
        count, tenantId, sw.ElapsedMilliseconds);   // structured, queryable fields
}

// Program.cs — OpenTelemetry: traces + metrics out of the box, exported to App Insights
builder.Services.AddOpenTelemetry()
    .WithTracing(t => t.AddAspNetCoreInstrumentation().AddSqlClientInstrumentation())
    .WithMetrics(m => m.AddAspNetCoreInstrumentation().AddRuntimeInstrumentation())
    .UseAzureMonitor();   // one request now traces across API → SQL → cache with timings

THE THREE PILLARS, TIED BY A CORRELATION ID

  Logs   -- "what happened"   ->  structured, searchable by {CorrelationId, TenantId}
  Metrics-- "how much/often"  ->  p95 latency, RPS, error rate, GC, queue length (alerts)
  Traces -- "where the time"  ->  API span -> SQL span -> cache span, with durations

  customer quotes correlation id  ->  one query finds the log, the trace, the slow span

Mattrx metric: adding structured logging + correlation IDs + OpenTelemetry traces took mean time to diagnose a production incident from ~35 minutes to ~4 minutes — you jump straight to the failing span instead of grepping. Alerting on the right metrics (p95, error rate, thread-pool queue) also caught two regressions in staging that log-watching would have missed.

---

Lesson 12 — AI is Becoming Part of Enterprise Architecture

In the last two years AI stopped being a side experiment and became a component in the architecture — with the same engineering rigor as any other dependency: interfaces, guardrails, evaluation, cost control. The mistake teams make is treating an LLM as a magic oracle wired straight to the user. The lesson: treat AI like an untrusted, probabilistic service — wrap it, ground it, validate its output, and measure it.

There are three distinct AI shapes I now design for, and they're not interchangeable:

AI IN ENTERPRISE ARCHITECTURE — three shapes, three jobs

1. RAG (retrieval-augmented generation)   ->  answer from YOUR docs, grounded + cited
   Mattrx Help: Semantic Kernel + Azure AI Search over the docs

2. Agentic (tools + reasoning loop)        ->  take actions, call typed functions
   Mattrx Insights: an agent that queries data and assembles analysis with tools

3. Classical ML (regression/classify/cluster) ->  predict numbers/categories/groups
   ML.NET in-process: churn, conversion forecast, segmentation (NO LLM needed)

Before — LLM wired naively to the user

// BEFORE — raw model call, no grounding, no validation. Hallucinations reach the customer.
var answer = await llm.CompleteAsync(userQuestion, ct);   // ungrounded, unverified
return answer;                                            // could be confidently wrong

After — AI as a guarded, grounded component

// AFTER — retrieve context, ground the prompt, validate the output, measure it
public async Task<HelpAnswer> AskAsync(string question, CancellationToken ct)
{
    var context = await search.RetrieveAsync(question, topK: 5, ct);   // ground in OUR docs
    if (context.Count == 0)
        return HelpAnswer.NoAnswer("I don't have docs on that.");       // refuse, don't hallucinate

    var prompt = Prompt.Grounded(question, context);                    // context-constrained
    var raw = await llm.CompleteAsync(prompt, ct);
    var answer = Guardrails.Validate(raw, context);                     // citations + safety check
    metrics.RecordAiCall(tokens: raw.Usage, grounded: true);           // cost + quality tracking
    return answer;                                                      // grounded, cited, measured
}

The architectural rules I now hold: ground generative answers in your own data (RAG, with a refuse-path), use classical ML when the problem is numbers (don't call an LLM to predict churn), validate and cite model output before it reaches a user, and measure tokens/latency/quality like any other dependency.

Mattrx metric: the grounded RAG help system (Mattrx Help) deflects ~520 support tickets/month at a fraction of a human's cost — but only because it's grounded and has a refuse-path; the naive ungrounded version hallucinated answers that created tickets. The classical-ML predictions (churn, forecasting) run in-process via ML.NET at ~3 ms with no LLM cost at all (full teardown linked below).

---

The thread through all three — and the whole series

Azure         ->  match platform to workload, lean on managed, right-size with autoscale
Observability ->  logs + metrics + traces, tied by a correlation id — see before you fix
AI            ->  a guarded, grounded, measured component — not a magic oracle

Fourteen years, twelve lessons, one underlying theme across all four parts: enterprise software rewards restraint and fundamentals over novelty. Clean boundaries (Part 1), a data layer that respects the database (Part 2), ceilings removed and architecture split only when warranted (Part 3), and infrastructure sized to reality with eyes-on observability and AI treated as an engineered component (Part 4). None of it is flashy. All of it is what keeps a system alive — and a team shipping — past year five.

---

You've reached the end of the series

That's all four parts of 14 Years of Enterprise ASP.NET. Thanks for reading the whole thing.

The full series:

1. Part 1: Clean Code, OOP & SOLID
2. Part 2: LINQ, EF Core & SQL Server
3. Part 3: Performance, Microservices & Design Patterns
4. Part 4 (you are here): Azure, Observability & AI

→ Start over at Part 1, or dive deeper via the links below.

---

Further reading

• Azure Deploy Targets in 2026: App Service vs Container Apps vs AKS — Lesson 10 as a decision guide.
• Production RAG with Azure OpenAI + Azure SQL — A Complete Guide for .NET Teams — the grounded RAG architecture from Lesson 12.
• No Python, No PhD: Train Real ML Models in C# with ML.NET — the classical-ML shape from Lesson 12.
• Scaling ASP.NET Core APIs to 100,000 Requests Per Minute — the performance and right-sizing work behind Lesson 10.

---

Standing up observability, sizing Azure, or wiring AI into an enterprise system without it going off the rails? Email randhir.jassal@gmail.com with where you are and I'll point at which of these lessons applies — and which to skip.