Lesson 29 of 30

Container Metrics & Observability

Q: Your team currently relies on `docker stats` to check container health. You're moving from a handful of containers on one host to a fleet spread across many machines. What breaks about that approach, and what would you put in place instead?

A strong answer covers: Identifies what docker stats can't do at fleet scale: single-host scope, real-time only with no history, no aggregation or alerting; Names cAdvisor (or an orchestrator's built-in equivalent) as the per-host agent collecting per-container metrics in an aggregatable form; Names Prometheus (scraping/storage) and Grafana (dashboards/alerting) as the standard next layer, or an equivalent alerting system; Connects metrics to resource limits — the point is noticing a container trending toward an OOM-kill before it happens, not just after.

`docker stats` answers 'is this one container healthy right now' — and stops being a usable answer the moment you have more containers than fit on one terminal screen. What replaces it is the same aggregation problem logging has, applied to CPU, memory, and I/O.

11 min read

Check your understanding

Every option explains itself — why the right one is right, and why each wrong one is wrong. Miss one and it comes back later on a spaced-repetition schedule.

Question 1 of 3

`docker stats` works well for checking one container's resource usage during local debugging. Why does it stop being a usable tool for a fleet of 200 containers across 15 machines?

It only shows the current host's currently-running containers in real time, with no history and no cross-host aggregation or alertingdocker stats has a hard-coded limit of showing at most 5 containers at oncedocker stats requires a paid Docker license once more than a few containers are running

Stuck? Ask the AI tutor

Hints without spoilers, and a plain explanation of why a wrong answer was wrong — answered only from this lesson. Free account, no card.

Interview questions on Container Metrics & Observability

What interviewers actually ask about this, and what a strong answer has to cover.

Your team currently relies on `docker stats` to check container health. You're moving from a handful of containers on one host to a fleet spread across many machines. What breaks about that approach, and what would you put in place instead?

A strong answer covers

Identifies what docker stats can't do at fleet scale: single-host scope, real-time only with no history, no aggregation or alerting
Names cAdvisor (or an orchestrator's built-in equivalent) as the per-host agent collecting per-container metrics in an aggregatable form
Names Prometheus (scraping/storage) and Grafana (dashboards/alerting) as the standard next layer, or an equivalent alerting system
Connects metrics to resource limits — the point is noticing a container trending toward an OOM-kill before it happens, not just after

Likely follow-ups

A container just got OOM-killed. Walk me through what you'd check first, and why metrics history matters more here than it might for a different kind of failure.
How would you decide what threshold should trigger an alert versus just being visible on a dashboard?

Practice these with the AI interviewer →

The plain-language idea

Metrics are numeric measurements sampled over time — CPU percent, memory usage, network I/O, disk I/O — as opposed to the discrete event records logging deals with. The resource-limits lesson in this track covers setting cgroup-enforced CPU/memory limits; this lesson is about observing what a container is actually using against those limits, and noticing it's approaching them before it gets OOM-killed rather than after.

docker stats is the direct, manual answer: a live terminal view of CPU%, memory usage/limit, network I/O, and block I/O for running containers on the current host. It's genuinely useful for a quick "is this one container behaving" check during local debugging — and it stops scaling almost immediately past that. It shows only the current host, only currently-running containers, only right now, with no history and no alerting. A fleet of 200 containers across 15 machines has no meaningful docker stats story at all.

cAdvisor (Container Advisor) is the piece that generalizes this: an agent that runs on each host, collecting the same category of per-container resource metrics docker stats shows, but exposing them in a form built for aggregation rather than a live terminal view. An orchestrator's own built-in metrics (Kubernetes' metrics-server, and similar) do an equivalent job, often using cAdvisor internally. Either way, the typical next step is the same: those metrics get scraped by Prometheus (a time-series metrics database built for exactly this kind of periodic scraping and storage) and visualized in Grafana (dashboards and alerting on top of that stored data) — the standard stack for "resource metrics across a whole fleet, queryable and alertable, not just visible one host at a time."

See it in a real thing: from one terminal to a fleet-wide dashboard

# manual, one host, right now
docker stats --no-stream
# CONTAINER   CPU %   MEM USAGE / LIMIT   MEM %   NET I/O
# api-1       340%    980MiB / 1GiB       95.7%   1.2MB / 800kB

That single row is the whole story docker stats can tell — a memory usage sitting at 95.7% of its limit, one step from an OOM-kill, visible only because someone happened to be watching this terminal at this moment. The aggregated version of the same fact — cAdvisor scraped by Prometheus, graphed and alerted on in Grafana — turns that "someone happened to notice" into "an alert fires automatically when any container in the fleet crosses a memory threshold," which is the actual point: metrics exist so you find out before the OOM-kill, not from its aftermath in the logs.

What people usually miss

Watch out

docker stats has no history. It shows the current instant only — if a container spiked to 100% CPU for thirty seconds five minutes ago and you weren't watching, that data is simply gone, which is precisely the gap a time-series store like Prometheus is built to close.
Metrics and logs answer different questions, the same way the observability lesson distinguishes monitoring from logging. Metrics tell you a container's memory climbed steadily over an hour before it was OOM-killed (the "what trend led here"); logs tell you what the application was actually doing in the seconds before the kill — you typically need both to fully diagnose an incident.
A container hitting its memory limit gets OOM-killed by the kernel, not gracefully shut down. There's no SIGTERM grace period in that path the way there is for an orchestrator-initiated stop — this is a hard kill, which is exactly why watching the metric trend beforehand matters more than reacting to the kill afterward.
Per-container metrics without per-container limits configured tell an incomplete story. A container using 95% of available host memory looks very different from one using 95% of an explicitly set 512MB limit — metrics are most actionable read alongside the resource limits they're measured against, not in isolation.

Why this matters in interviews

"A container keeps getting killed in production and we don't know why" is a real incident-response scenario, and a strong answer routes through metrics history (was memory climbing before the kill, was it CPU throttling) rather than only checking logs after the fact — naming cAdvisor/Prometheus/Grafana as the standard fleet-wide answer, and docker stats as its useful-but-limited single-host precursor, shows the difference between debugging one container and operating a fleet of them.