MCP Deep Dive, Part 13: Hosting MCP on Azure at Real Scale — Container Apps, Autoscaling, and the SSE Gotcha

A local MCP server is a dotnet run. A production one is a stateless HTTP service that also holds long-lived streams, gets hammered at 3,200 requests a second, deploys without dropping a call, and never leaks a connection string. That's an infrastructure problem, and on Azure it has a specific — and mostly pleasant — answer, plus one gotcha that will bite you exactly once. This part is that answer.

This is Part 13 of a 15-part deep dive on Model Context Protocol (MCP). Part 12 built the server in .NET; now we run it. The three Mattrx servers — mattrx-analytics, mattrx-reports, mattrx-admin — live on Azure Container Apps behind a gateway, and this part is how: hosting, autoscaling, the SSE-streaming gotcha, zero-downtime deploys, and secretless security.

TL;DR

Concern	Naive hosting (before)	Azure at scale (after)
Compute	fixed VM / always-on App Service	Container Apps (serverless, revisions)
Scaling	fixed replicas	KEDA HTTP autoscale (+ scale-to-zero dev)
Edge	servers exposed directly	Front Door / APIM gateway (Part 11)
Streaming	SSE dies behind the LB	affinity + keepalive + raised timeout
Deploys	in-place, drops calls	revisions + canary + readiness
Secrets / network	connection strings, public	managed identity, Key Vault, private

• Host MCP servers on Azure Container Apps — serverless containers with revisions, ingress, and KEDA autoscaling.
• Autoscale by HTTP concurrency: min 2 warm replicas, scale to ~30 at the 3,200 rps peak; scale-to-zero for dev/preview.
• Put the org gateway (Front Door / APIM) in front — TLS, Entra pre-check (Part 6), rate-limit, path routing to the 3 servers (Part 11).
• The SSE gotcha: streaming (Part 9) dies behind the load balancer unless you raise the ingress idle timeout, enable session affinity, and send keepalive pings.
• Zero-downtime deploys via Container Apps revisions + traffic splitting + readiness probes (Part 3) — canary, no dropped calls.
• Managed identity to Entra + Azure SQL — no connection strings; secrets via Key Vault; internal ingress so only the gateway reaches the servers.
• Azure SQL behind a private endpoint; observability to App Insights (Part 10).
• Native AOT (Part 12) keeps cold starts fast, so autoscaling stays snappy.
• ~85k tool calls/day, read p95 120 ms, streaming first-token ~300 ms — on this infra.
• The MCP servers moved off the old App Service (P1v3×2 + burst) onto Container Apps for elastic, per-server scale.

The one mental shift: an MCP server on Azure is a stateless HTTP service that also holds long-lived streams. Container Apps gives you elastic scale, revisions, and internal ingress; a gateway gives you the edge; managed identity gives you secretless security. The one thing that will bite you is SSE behind the load balancer — get affinity, keepalive, and timeouts right and streaming just works.

The Azure topology

                    Agents / clients
                          |
              [ Azure Front Door / APIM ]  <- org MCP gateway (Part 11)
                 TLS · Entra pre-check (Part 6) · rate-limit · route by path
                          |
        +-----------------+------------------+
        |                 |                  |
        v                 v                  v
  Container Apps environment  (managed identity, internal ingress)
   mattrx-analytics    mattrx-reports    mattrx-admin
   min 2 / max 30      (-> Service Bus)   (locked)
        |                 |                  |
        +--------+--------+---------+--------+
                 |                  |
                 v                  v
          Azure SQL (private)   Key Vault · Service Bus · App Insights (Part 10)

1. Compute: Azure Container Apps

Before

A fixed VM or an always-on App Service plan — over-provisioned at 3am, under-provisioned at the 9am peak, and one shared plan for three very different servers.

After

Each MCP server is an Azure Container App: a serverless container with built-in ingress, revisions, and KEDA autoscaling, in a shared environment.

// Azure Container App hosting mattrx-analytics — serverless containers, scale rules, revisions.
resource analytics 'Microsoft.App/containerApps@2024-03-01' = {
  name: 'mattrx-analytics'
  identity: { type: 'SystemAssigned' }             // managed identity (section 6)
  properties: {
    managedEnvironmentId: env.id
    configuration: {
      ingress: { external: false, targetPort: 8080, transport: 'http', allowInsecure: false }
    }
    template: {
      containers: [ { name: 'server', image: 'mattrxacr.azurecr.io/mcp-analytics:2.4.0' } ]
      scale: { minReplicas: 2, maxReplicas: 30 }   // rules below
    }
  }
}

Diagnostic: Container Apps fits MCP servers precisely — they're stateless HTTP services that need to scale independently. Three servers, three apps, one environment: mattrx-analytics scales on read load, mattrx-reports barely scales (it just enqueues), mattrx-admin stays tiny. One shared plan could never balance those three.

Mattrx metric: moving the servers off the old App Service (P1v3×2 + B2 burst) onto Container Apps gave each server its own elastic scale — the analytics server absorbs the peak while the admin server stays at two replicas.

2. Autoscaling with KEDA

Before

A fixed replica count: pick too low and you fall over at peak, pick too high and you pay for idle capacity all night.

After

A KEDA HTTP scale rule adds replicas by concurrency, with a warm floor and a hard ceiling — and scale-to-zero for non-prod.

scale: {
  minReplicas: 2                 // keep warm; AOT (Part 12) makes even cold starts fast
  maxReplicas: 30
  rules: [ {
    name: 'http-concurrency'
    http: { metadata: { concurrentRequests: '100' } }   // +1 replica per ~100 concurrent requests
  } ]
}
// Dev / preview environment: minReplicas: 0 -> scale-to-zero, $0 when idle.

Diagnostic: MCP load is bursty — campaigns end on the hour, agents fan out — so scale by concurrency, not CPU. Keep a warm floor (minReplicas: 2) so the first request of a burst isn't a cold start, cap maxReplicas so a runaway agent can't scale you into a surprise bill, and set minReplicas: 0 in dev to pay nothing when idle.

Mattrx metric: the analytics server sits at 2 replicas overnight and scales toward ~30 at the ~3,200 rps peak, serving read-tool p95 120 ms throughout; dev and preview environments scale to zero and cost nothing between test runs.

3. The gateway in front

Before

Agents hit the servers directly — no central TLS, auth pre-check, rate-limit, or routing, and the servers were publicly exposed.

After

The org gateway from Part 11 — Azure Front Door or API Management — sits in front of the environment as the one public, governed edge.

  Agents ---> [ Azure Front Door / APIM ]  (the org MCP gateway, Part 11)
                - TLS termination
                - Entra token pre-validation (Part 6)
                - rate-limit per tenant / connector
                - route by path:  /analytics -> mattrx-analytics
                                  /reports   -> mattrx-reports
                                  /admin     -> mattrx-admin
                    |
                    v
              Container Apps environment (internal ingress — servers not public)

Diagnostic: the gateway is where cross-cutting edge concerns live — TLS, a first-pass token check (Part 6), per-tenant rate limits (Part 11), and path-based routing to the three servers. Because it fronts everything, the servers keep internal ingress and stay identical — no server re-implements the edge.

Mattrx metric: one gateway routes all traffic to the three internal servers; the servers never see the public internet, and rate-limits at the edge are what stop a runaway agent from turning autoscaling into a cost spike.

4. The SSE gotcha — streaming behind the load balancer

Before

A streaming tool (Part 9) works perfectly on localhost, ships to Azure, and then "randomly" dies in production — connections drop mid-stream, or a stream loses its state.

After

Configure the ingress for long-lived SSE: raise the idle timeout, enable session affinity for stateful streams, and send keepalive pings.

ingress: {
  external: false
  targetPort: 8080
  transport: 'http'                          // match what your SSE client expects (HTTP/1.1 chunked)
  stickySessions: { affinity: 'sticky' }     // pin a stateful SSE session to one replica
}
// + server keepalive pings every ~15s (Part 9) so the ingress never sees the stream as "idle".

Diagnostic: this is the number-one Azure MCP surprise. SSE (Part 9) holds a connection open and sends events sparsely — which the Container Apps ingress and Front Door read as idle and reap, and load-balance mid-stream to a replica that doesn't hold the session. It only fails in Azure, never locally, so it ambushes you in prod. The fix is three settings: raise the ingress idle timeout, turn on session affinity for stateful streams, and keepalive-ping from the server. Miss any one and streaming breaks intermittently.

Mattrx metric: with affinity + keepalive, multi-minute streams and progress channels survive the gateway and ingress intact — streaming first-token p95 stays ~300 ms and long report streams don't drop.

5. Zero-downtime deploys with revisions

Before

An in-place deploy restarts the app and drops every in-flight tool call and SSE stream.

After

Container Apps revisions give you blue-green and canary — a new revision must pass its readiness probe (Part 3) before it takes any traffic, and you shift weight gradually.

# Roll out a new revision; canary 10%, watch, then shift to 100% — no dropped calls.
az containerapp update -n mattrx-analytics --image mattrxacr.azurecr.io/mcp-analytics:2.5.0

az containerapp ingress traffic set -n mattrx-analytics \
   --revision-weight latest=10 mcp-analytics--2-4-0=90     # 10% canary on the new revision

// Readiness gates traffic: an unhealthy revision never receives a request (Part 3 /readyz).
probes: [ { type: 'Readiness', httpGet: { path: '/readyz', port: 8080 } } ]

Diagnostic: revisions make MCP deploys boring in the best way. The new revision boots, its /readyz (Part 3) must pass, then you canary 10% and watch the per-tool metrics (Part 10) before going to 100%. The old revision keeps serving in-flight calls and streams until traffic drains — no restart, no dropped work.

Mattrx metric: deploys are zero-downtime — in-flight tool calls and SSE streams finish on the old revision while the new one ramps, so shipping a server update never shows up as an error blip.

6. Secretless security — managed identity, Key Vault, private

Before

Connection strings for Azure SQL and Service Bus sat in app config, and the servers had public ingress.

After

A managed identity authenticates to Entra (Part 6), Azure SQL, and Service Bus — no connection strings. Any remaining secrets come from Key Vault by reference, and ingress is internal.

identity: { type: 'SystemAssigned' }                 // -> Entra, Azure SQL, Service Bus, no secrets
secrets: [ { name: 'sb-conn', keyVaultUrl: kv.properties.vaultUri, identity: 'system' } ]
// Azure SQL reached over a PRIVATE ENDPOINT; ingress external:false -> only the gateway can reach it.

Diagnostic: this closes the loop on every security lesson in the series. The servers authenticate with a managed identity, so there are no connection strings to leak (the exact failure the Security and Auth parts warned about). Secrets that must exist come from Key Vault by reference, Azure SQL sits behind a private endpoint, and internal ingress means only the gateway is reachable from outside.

Mattrx metric: zero connection strings in configuration across the three servers — identity to Azure SQL and Service Bus is a managed identity, which also means credential rotation is Azure's job, not ours.

Scale and streaming, at a glance

  KEDA HTTP scale rule:  +1 replica per ~100 concurrent requests
     idle .......... 2 replicas (min, warm)
     peak 3,200 rps  ~30 replicas
     dev / preview .. 0 replicas (scale-to-zero, $0 idle)

  SSE stream:  client -- keepalive every ~15s --> Front Door -- affinity --> same replica
               (raise ingress idle timeout; without keepalive + affinity, the stream dies in Azure)

The numbers, in one place

Aspect	Naive hosting (before)	Container Apps (after)
Idle cost (dev)	pays 24/7	$0 (scale-to-zero)
Peak handling	fixed, falls over	autoscale to ~30 replicas
Read-tool p95	varies	120 ms through peak
Streaming	breaks behind the LB	survives (affinity + keepalive)
Deploys	drop in-flight calls	zero-downtime (revisions)
Secrets in config	connection strings	0 (managed identity)
Server exposure	public	internal (gateway only)

Azure hosting checklist

• Run each MCP server as its own Container App in a shared environment.
• Autoscale by HTTP concurrency (KEDA); warm floor in prod, scale-to-zero in dev.
• Cap maxReplicas; rely on gateway rate-limits (Part 11) so scale-out isn't a cost-out.
• Front everything with Front Door / APIM; keep server ingress internal.
• For SSE: raise the ingress idle timeout, enable session affinity, send keepalive (Part 9).
• Deploy with revisions + traffic splitting; gate on /readyz (Part 3).
• Use a managed identity for Entra / Azure SQL / Service Bus — no connection strings.
• Key Vault for any real secrets; private endpoint to Azure SQL; App Insights (Part 10).

The honest stuff: proportion and pitfalls

1. A single low-traffic server. App Service — or a small VM — is simpler. Container Apps + a gateway earns its keep with multiple servers and elastic scale.
2. Scale-to-zero in prod for latency-sensitive servers. Even an AOT cold start adds latency. Keep minReplicas ≥ 1–2 in production; scale-to-zero is a dev/preview trick.
3. Skipping the SSE settings. The most common Azure MCP failure, and it only shows up in Azure. Test streaming through the real gateway, not just localhost.
4. Session affinity everywhere. Only stateful SSE needs it; stateless tool calls scale better without it. Don't pin what you don't have to.
5. Connection strings in config. Use managed identity — a leaked string is the breach every security part warned about.
6. Public server ingress. Keep the servers internal; only the gateway is public. A directly-reachable server bypasses your edge.
7. Unbounded maxReplicas. A runaway agent loop can spike concurrency into a huge bill. Cap it and lean on gateway rate-limits.

The model to carry forward

An MCP server on Azure is a stateless HTTP service that also holds long-lived streams — host it as both. Container Apps gives you elastic, per-server scale, revisions for zero-downtime, and internal ingress; a Front Door / APIM gateway gives you one governed public edge; a managed identity gives you secretless security. And the single thing that will surprise you is SSE behind the load balancer — configure affinity, keepalive, and timeouts, and it disappears.

Three habits for hosting MCP on Azure:

1. Scale by concurrency; keep prod warm. KEDA HTTP rules with a warm floor; scale-to-zero only for dev.
2. One gateway in front; servers internal. A single public, rate-limited, secretless edge — nothing else reaches the servers.
3. Prove streaming in Azure, not localhost. The SSE settings only fail behind the real load balancer, so test them there.

In Part 14 we widen the lens past .NET and Azure: wiring MCP into OpenAI and other agent frameworks — because the servers we've built should drive any model, not just ours.

Continue the series — MCP Deep Dive

1. Why Model Context Protocol Kills Integration Glue Code for Good
2. Inside the MCP Architecture: Hosts, Clients, and Servers
3. Build a Production-Grade MCP Server From Scratch
4. Build an MCP Client That Connects to Any Tool (and Any Model)
5. Custom MCP Tools Your AI Agents Can Actually Trust
6. MCP Authentication With OAuth and Entra ID, Done Right
7. Reaching a Tool Isn't Being Allowed — Least-Privilege Authorization for MCP Agents
8. When a Tool Result Is the Attack — Securing MCP Against Prompt Injection and Tool Abuse
9. When the Tool Takes Minutes — Streaming and Long-Running Tools Over MCP
10. When the Agent Feels Off — Debugging and Observability for MCP in Production
11. The Protocol Was Never the Blocker — Rolling MCP Across the Enterprise
12. Building MCP Servers in C# and .NET 9 — The SDK, DI, and Native AOT
13. Hosting MCP on Azure at Real Scale — Container Apps, Autoscaling, and the SSE Gotcha (you are here)
14. Wiring MCP Into OpenAI and Agent Frameworks
15. Running MCP in Production — Lessons From Mattrx

Further reading

• MCP Deep Dive, Part 9: When the Tool Takes Minutes — Streaming and Long-Running Tools Over MCP
• MCP Deep Dive, Part 12: Building MCP Servers in C# and .NET 9
• Azure App Service vs Container Apps vs AKS in 2026: Which to Pick

---

Hosting MCP servers on Azure and want a second pair of eyes on the scale rules or the SSE settings? I'm always happy to compare notes — reach me at randhir.jassal@gmail.com.