Lesson 12 of 30

Health Checks & Restart Policies

Q: Your service occasionally stops responding to requests but the process never crashes — `docker ps` keeps showing it as running the whole time. How would you make this failure mode detectable and self-healing?

A strong answer covers: Distinguish 'running' (process alive) from 'healthy' (actually serving requests) and explain why a restart policy alone can't catch this, since it only reacts to process exit; Propose a HEALTHCHECK / Compose healthcheck: hitting a real readiness endpoint on an interval, with a sensible retries threshold; Explain that Docker marking a container unhealthy is only a status change — something else (an orchestrator, or the app exiting on its own health failure) has to act on it to actually restart; Mention start_period to avoid false unhealthy status during normal startup.

"Running" and "working" are different claims — a health check is what closes that gap, and a restart policy is what decides what happens once you know the difference.

10 min read

Check your understanding

Every option explains itself — why the right one is right, and why each wrong one is wrong. Miss one and it comes back later on a spaced-repetition schedule.

Question 1 of 3

A container's process hangs on a deadlocked database call but never exits. With no HEALTHCHECK defined, what does `docker ps` show, and what does a restart policy do about it?

docker ps still shows it as running, and no restart policy triggers, since the process never exiteddocker ps shows it as unhealthy, and an on-failure restart policy restarts it automaticallyDocker detects the hang via its default liveness probe and restarts the container regardless of the configured restart policy

Stuck? Ask the AI tutor

Hints without spoilers, and a plain explanation of why a wrong answer was wrong — answered only from this lesson. Free account, no card.

Interview questions on Health Checks & Restart Policies

What interviewers actually ask about this, and what a strong answer has to cover.

Your service occasionally stops responding to requests but the process never crashes — `docker ps` keeps showing it as running the whole time. How would you make this failure mode detectable and self-healing?

A strong answer covers

Distinguish 'running' (process alive) from 'healthy' (actually serving requests) and explain why a restart policy alone can't catch this, since it only reacts to process exit
Propose a HEALTHCHECK / Compose healthcheck: hitting a real readiness endpoint on an interval, with a sensible retries threshold
Explain that Docker marking a container unhealthy is only a status change — something else (an orchestrator, or the app exiting on its own health failure) has to act on it to actually restart
Mention start_period to avoid false unhealthy status during normal startup

Likely follow-ups

If this container runs under plain `docker run --restart always` with no orchestrator, does marking it unhealthy actually cause a restart? What would you need to add to make it self-heal?
How would you design the /health endpoint itself — should it check the app's own liveness only, or also its downstream dependencies like a database connection? What's the tradeoff?

The plain-language idea

docker ps showing Up 3 minutes only means the container's main process hasn't exited. It says nothing about whether that process is actually serving requests, connected to its database, or hung on a deadlock while technically still alive. A health check is an explicit, repeated probe — a command Docker runs on a schedule inside the container — that turns "the process is alive" into a real, checkable "the process is working" signal.

Declared in a Dockerfile with HEALTHCHECK, or in Compose with a healthcheck: block, it runs a command (often a curl against a /health endpoint, or a lightweight script) on an interval, and classifies the container as healthy, unhealthy, or starting based on the exit code and how many consecutive failures have happened. Without one, Docker's only signal for "is this container okay" is whether the process has exited at all — which misses the entire category of a hung-but-running process.

A restart policy is the separate, complementary decision: what should Docker do when a container's process actually exits? no (the default) does nothing. on-failure restarts only on a non-zero exit code, optionally capped at a max retry count. always restarts unconditionally, even after a clean manual stop, until you explicitly stop it again. unless-stopped is like always but respects an explicit docker stop across daemon restarts too. These two mechanisms are related but distinct: a restart policy reacts to the process exiting; a health check can detect a process that never exits at all but stopped doing useful work.

See it in a real thing: a healthcheck that catches what an exit code can't

FROM node:20-slim
COPY . .
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD curl -f http://localhost:3000/health || exit 1
CMD ["node", "server.js"]

services:
  api:
    build: .
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      retries: 3

If server.js hangs — say, its event loop deadlocks on a stuck database call — the process never exits, so a restart policy alone would never trigger. docker ps would still show it as running. The HEALTHCHECK is what actually notices: three consecutive failed probes flips the container's status to unhealthy, which is the signal an orchestrator (or a human running docker ps) can act on, even though nothing about the process itself ever crashed.

What people usually miss

Watch out

A health check and a restart policy solve different problems and both matter. A restart policy alone misses hung-but-running processes; a health check alone (with restart: no) just reports the problem without fixing it. Production setups need both — the check to detect, the policy or orchestrator to act.
always restarts even after a deliberate stop, if the daemon itself restarts. docker stop on an always-policy container does stop it for that session, but if the Docker daemon or host reboots, always brings it back regardless of that manual stop — unless-stopped is usually the safer default for exactly this reason.
The starting health status exists to avoid false negatives during boot. A container needs time to actually initialize before its health endpoint is meaningful; --start-period (Dockerfile) or start_period: (Compose) gives it a grace window where failures don't count against the retry threshold yet.
on-failure with no retry cap can restart-loop forever on a genuinely broken container, burning CPU and flooding logs — setting on-failure:5 (or similar) is what turns an infinite crash loop into a bounded, diagnosable failure instead.

Why this matters in interviews

"How does an orchestrator know a container needs to be restarted?" is a common follow-up once a candidate has described self-healing in the abstract, and the strong answer names the health check as the actual detection mechanism — a restart policy alone can't catch a hung process, only a crashed one. It's also a direct, practical extension of "why did requests drop during a deploy," since a rolling update that waits on health checks behaves very differently from one that only waits on the container starting.