Lesson 2 of 14

Meta Robots Directives Beyond noindex

Q: Beyond noindex and nofollow, what other robots directives exist, and how would you apply indexing control to a resource that isn't an HTML page at all, like a PDF?

A strong answer covers: Distinguishes indexability directives (noindex) from content-display directives (max-snippet, max-image-preview, nosnippet, noarchive); Explains that conflicting directives resolve to the most restrictive one, not an average or a default; Names X-Robots-Tag as the HTTP-header equivalent for non-HTML resources (PDFs, images, videos) that have no to hold a meta tag; Mentions data-nosnippet as a fragment-level snippet exclusion, distinct from page-level indexing control.

Every developer knows noindex and nofollow. Almost none know that the robots meta tag has a second layer — max-snippet, max-image-preview, notranslate, data-nosnippet — that controls exactly how much of an indexed page a search engine is allowed to show, or that HTML isn't the only place these directives live.

12 min read

Check your understanding

Every option explains itself — why the right one is right, and why each wrong one is wrong. Miss one and it comes back later on a spaced-repetition schedule.

No quiz for this one — it's a practical topic

This topic is about judgment and practical tradeoffs more than a single right answer — a mock interview is the better way to test yourself on it.

Be interviewed on this →

Stuck? Ask the AI tutor

Hints without spoilers, and a plain explanation of why a wrong answer was wrong — answered only from this lesson. Free account, no card.

Interview questions on Meta Robots Directives Beyond noindex

What interviewers actually ask about this, and what a strong answer has to cover.

Beyond noindex and nofollow, what other robots directives exist, and how would you apply indexing control to a resource that isn't an HTML page at all, like a PDF?

A strong answer covers

Distinguishes indexability directives (noindex) from content-display directives (max-snippet, max-image-preview, nosnippet, noarchive)
Explains that conflicting directives resolve to the most restrictive one, not an average or a default
Names X-Robots-Tag as the HTTP-header equivalent for non-HTML resources (PDFs, images, videos) that have no <head> to hold a meta tag
Mentions data-nosnippet as a fragment-level snippet exclusion, distinct from page-level indexing control

Likely follow-ups

How would you noindex an entire directory of PDFs without touching each file individually?
Why might a site want max-image-preview:large but noarchive on the same page — what's the business reasoning?

Practice these with the AI interviewer →

The plain-language idea

<meta name="robots" content="noindex, nofollow"> is the directive every frontend developer eventually learns — keep this page out of the index, don't pass link equity through it. But noindex/nofollow only answer whether a page can appear in search results at all. A separate, much less commonly known set of directives controls something different: once a page is indexed, how much of it Google is allowed to show in the result — the snippet text, the image preview, the cached copy, even whether a URL should self-destruct from the index on a schedule.

These all go in the same content attribute, comma-separated, and can be combined:

max-snippet:[number] — caps the text snippet's length in characters. max-snippet:0 means no text snippet at all (functionally the same as nosnippet for text).
max-image-preview:[none|standard|large] — caps how large an image preview can be shown for the page.
max-video-preview:[number] — caps an animated video preview's duration in seconds; 0 means a static thumbnail only.
nosnippet — blanket-blocks any text snippet or video preview, full stop.
noarchive — removes the cached-page link Google can otherwise show for a result.
unavailable_after:[date] — treat the page as noindex automatically once the given date (ISO 8601) passes, without needing a deploy on that day.
notranslate — tells Google not to offer an automatic translation of this page in results.

When directives conflict, the more restrictive one wins — a page with both max-snippet:50 and nosnippet gets no snippet at all, not a 50-character one.

See it in a real thing: a paywalled article that still wants to rank

<meta name="robots" content="max-snippet:150, max-image-preview:large, noarchive">

This article wants to be indexed and shown with a generous snippet and a large image preview (both help click-through), but doesn't want Google offering a cached copy — maybe because the cache would expose paywalled content that should only be visible through the site's own metered-access logic.

For non-HTML resources — a PDF, an image, a video file — there's no <head> to put a <meta> tag in, so the same directives move to an HTTP response header instead:

X-Robots-Tag: noindex, nofollow

set from server config (Nginx, Apache, a Next.js middleware response header, a CDN edge rule) rather than markup. Any directive valid in the meta tag is valid in X-Robots-Tag, and it can even be scoped per-crawler:

X-Robots-Tag: googlebot: noindex, notranslate

And for excluding just part of a page from snippets — not the whole thing — there's data-nosnippet, an HTML attribute (not a meta directive) applied directly to the element you want hidden from snippet text:

<article>
  <p>This paragraph is safe to quote in a search snippet.</p>
  <p data-nosnippet>This paragraph — maybe pricing that changes daily — should never be.</p>
</article>

Watch out

noindex and nosnippet solve different problems — noindex keeps a page out of the index entirely; nosnippet lets it rank and appear, just without a preview of its content. Conflating them leads to either over-blocking pages that should rank, or under-blocking content you didn't want previewed.
The most restrictive directive always wins on conflict — stacking max-snippet:100 with nosnippet doesn't average out to something in between; nosnippet fully overrides it.
HTML isn't the only place these directives live. A PDF, a product image, or a video file has no <head> — X-Robots-Tag is the only way to apply these directives to non-HTML resources, and it's set at the server/CDN level, not in markup a frontend developer would normally touch.
unavailable_after removes deploy-day dependency for time-limited content — a limited-time promo page or an expiring job listing can be told to fall out of the index on schedule without anyone needing to ship a code change on that exact date.
data-nosnippet is fragment-level, not page-level, and it doesn't touch indexing — it only prevents that specific marked-up text from being quoted in the SERP snippet; the content is still indexed and the page can still rank on it.

Why this matters in interviews

Most candidates can recite noindex/nofollow from memory, which is exactly why it's a weak signal on its own — an interviewer who asks "how would you let a page rank but stop Google from showing a cached copy or an oversized image preview" is deliberately probing past the memorized pair to see whether you understand the content directives (max-snippet, max-image-preview, noarchive) as a separate axis from indexability, and whether you know X-Robots-Tag exists for the non-HTML resources a <meta> tag literally can't reach.