Lesson 12 of 30

Elastic Load Balancing

Q: You have three backend instances behind a load balancer, and one of them starts throwing errors under load. Walk through what happens automatically, and how you'd choose between an ALB and an NLB for this service in the first place.

A strong answer covers: Explains that health checks continuously probe each target and, after a sustained failure threshold, mark it unhealthy and remove it from rotation without human intervention; Explains that the remaining healthy targets absorb the redistributed traffic; Names the real ALB vs NLB tradeoff — layer 7 content-aware routing (path/host) versus layer 4 raw throughput/latency — rather than treating one as a strict upgrade over the other; Mentions the target is added back automatically once it passes health checks again.

A load balancer's real job isn't spreading traffic evenly — it's noticing a target has gone bad and quietly routing around it before a human ever finds out.

11 min read

Check your understanding

Every option explains itself — why the right one is right, and why each wrong one is wrong. Miss one and it comes back later on a spaced-repetition schedule.

Question 1 of 3

A service needs to route `/api/*` requests to one set of backend targets and `/admin/*` requests to a different set, through the same load balancer. Which type of load balancer supports this?

ALB — it operates at layer 7 and can route based on the request's pathNLB — it operates at a lower layer, which gives it more routing flexibilityBoth support this equally — the choice only affects throughput, not routing capability

Stuck? Ask the AI tutor

Hints without spoilers, and a plain explanation of why a wrong answer was wrong — answered only from this lesson. Free account, no card.

Interview questions on Elastic Load Balancing

What interviewers actually ask about this, and what a strong answer has to cover.

You have three backend instances behind a load balancer, and one of them starts throwing errors under load. Walk through what happens automatically, and how you'd choose between an ALB and an NLB for this service in the first place.

A strong answer covers

Explains that health checks continuously probe each target and, after a sustained failure threshold, mark it unhealthy and remove it from rotation without human intervention
Explains that the remaining healthy targets absorb the redistributed traffic
Names the real ALB vs NLB tradeoff — layer 7 content-aware routing (path/host) versus layer 4 raw throughput/latency — rather than treating one as a strict upgrade over the other
Mentions the target is added back automatically once it passes health checks again

Likely follow-ups

If that failing target had requests already in flight when it was marked unhealthy, what happens to those specific requests?
How would your answer change if this were a UDP-based service instead of an HTTP API?

Practice these with the AI interviewer →

The plain-language idea

Elastic Load Balancing (ELB) distributes incoming traffic across multiple backend targets (EC2 instances, containers, IP addresses) so no single one gets overwhelmed, and so traffic never gets sent to a target that's already broken. Two load balancer types cover almost every real use case, and the choice between them is a genuine tradeoff:

ALB (Application Load Balancer) — operates at layer 7 (HTTP/HTTPS). It actually reads the request and can route based on its content: path (/api/* to one set of targets, /admin/* to another), hostname, or headers, all through the same load balancer.
NLB (Network Load Balancer) — operates at layer 4 (TCP/UDP). It forwards raw connections without inspecting them, which makes it capable of extremely high throughput and very low, predictable latency — the tradeoff for not understanding HTTP at all.

Targets are organized into target groups — a named set of registered targets the load balancer routes to, each with its own health check configuration: a periodic probe (an HTTP path like /health, or a raw TCP handshake) that decides whether a target is fit to receive traffic right now.

See it in a real thing: a target fails its health check mid-traffic

Health checks pull a bad target out of rotationStep 1 of 4

A load balancer routes traffic evenly across three healthy targets.

An ALB routes requests evenly across three targets in a target group. One of them starts failing its health check — maybe its process is deadlocked, maybe a dependency it needs is down. After enough consecutive failures cross the configured threshold, the ALB marks it unhealthy and stops routing new requests to it, automatically, with no human involved. The other two targets absorb the traffic that would have gone to it. When the failing target recovers and starts passing health checks again, it's added back into rotation the same way it was removed — automatically.

What people usually miss

Watch out

ALB and NLB aren't "new vs. old" — they're a real layer tradeoff. ALB's content-aware routing (path/host-based rules) is only possible because it operates at layer 7 and actually parses HTTP; NLB's speed and near-zero latency come specifically from not doing that. Picking one is about what the workload needs, not defaulting to whichever sounds more modern.
A single failed health check usually doesn't pull a target out of rotation. Health checks use a configured failure threshold (e.g. 3 consecutive failures) specifically so one slow response or transient blip doesn't cause unnecessary churn — the same "sustained breach, not one data point" logic as a CloudWatch alarm.
Removing a target from rotation doesn't kill its existing connections. Deregistration (or marking unhealthy) stops new requests from being routed there; in-flight requests can be allowed to finish via a configurable deregistration delay — an abrupt cutoff isn't the default behavior.
Health checks are configured per target group, not per load balancer. One ALB can front several target groups (different services, different routes), each with its own independent health check path and thresholds — there's no single global health check for the whole load balancer.

Why this matters in interviews

"How does traffic get routed away from a broken instance without anyone paging on-call?" tests whether a candidate understands health checks as an automatic, threshold-based mechanism baked into the load balancer — not something that requires a monitoring script or a human pulling a server out of a pool by hand.