Retry Logic and Backoff in the Data Layer

The plain-language idea

The earlier system-design coverage of resilience explains why a flaky dependency should be retried with increasing delay instead of hammered immediately. This lesson is the actual code: what a retry wrapper looks like, and the details that separate a wrapper that helps from one that quietly makes things worse.

A working retry wrapper needs three pieces:

• An increasing delay between attempts. Roughly doubling the wait each time — something like 500ms, then 1s, then 2s, then 4s — gives a struggling dependency room to recover instead of getting hit again immediately. The exact multiplier and starting delay aren't a fixed law, just a shape: each retry waits noticeably longer than the last.
• A maximum attempt count. A request that's never going to succeed shouldn't retry forever. Three to five attempts, then give up and surface the failure, is a reasonable default — "reasonable" depending on how long the user is willing to wait.
• A check for whether the failure is even worth retrying. This is the detail that's easy to skip and expensive to skip.

See it in a real thing: a fetch wrapper with backoff and a retryability check

async function fetchWithRetry(
  url: string,
  options: RequestInit = {},
  maxAttempts = 4,
): Promise<Response> {
  let attempt = 0;

  while (true) {
    attempt++;
    try {
      const res = await fetch(url, options);

      // A response came back — but "came back" isn't the same as "worth retrying on failure."
      if (res.ok) return res;
      if (!isRetryableStatus(res.status) || attempt >= maxAttempts) return res;
    } catch (networkError) {
      // A thrown error here means the request never completed (offline, DNS, timeout) —
      // that's a genuinely transient condition, so it's always worth retrying.
      if (attempt >= maxAttempts) throw networkError;
    }

    const baseDelay = 500 * 2 ** (attempt - 1); // 500ms, 1s, 2s, 4s...
    const jitter = Math.random() * baseDelay * 0.3; // up to 30% random variation
    await sleep(baseDelay + jitter);
  }
}

function isRetryableStatus(status: number): boolean {
  // 429 (rate limited) and 503 (service unavailable) are transient — the
  // server is asking for a slowdown, not rejecting the request as invalid.
  if (status === 429 || status === 503) return true;
  // 500-range errors other than 501 (not implemented) are usually transient too.
  if (status >= 500 && status !== 501) return true;
  // 400, 404, 401, 403 and similar: the request itself is the problem.
  // Retrying sends the exact same broken request and gets the exact same result.
  return false;
}

function sleep(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

The isRetryableStatus check is the part a naive retry wrapper skips. A 404 Not Found or 400 Bad Request will fail identically on every retry, because the problem is the request itself, not a temporary condition on the server. Retrying those anyway wastes several seconds doing something that was never going to succeed, and from the user's side it looks exactly like the app hanging.

What people usually miss

Watch out

• Not every failure deserves a retry. A 400/404/401 means the request itself is wrong — retrying it burns time and makes the failure feel slower and more confusing, not more resilient.
• "Exponential" doesn't mean one specific formula. Doubling is a common, reasonable shape, but the actual starting delay and multiplier are a judgment call based on how long a user will tolerate waiting, not a value to copy verbatim from somewhere.
• No maximum attempt count turns a real outage into an infinite loop. A backoff wrapper without a cap will retry a genuinely dead dependency forever, tying up whatever called it.
• Jitter isn't decoration. Without it, synchronized clients retry in lockstep and can regenerate the exact load spike the backoff was meant to avoid.
• Retries change idempotency requirements. Retrying a GET is safe by default; retrying a POST that creates something can create it twice unless the request is designed to be safe to repeat (an idempotency key, or checking whether the effect already happened).

Why this matters in interviews

"Would you retry this failed request, and how?" is a common follow-up to any resilience discussion, and the strong answer distinguishes retryable failures from non-retryable ones by name, not just recites "exponential backoff" as a phrase.