robots.txt for images
robots.txt can control how image crawlers like Googlebot-Image fetch your images. This page explains how to allow or disallow image crawling, the trade-off with Google Images visibility, and why blocking images for search is different from blocking pages.
Targeting image crawlers
Google uses Googlebot-Image to crawl images for Google Images. To stop image crawling of a directory while leaving page crawling intact, target the image token:
User-agent: Googlebot-Image Disallow: /private-images/
Blocking Googlebot-Image on a path means those images will not be newly indexed for image search. Other image crawlers (for example Bing's) use their own tokens, so target each one you care about.
- Googlebot-Image is Google's image crawler token
- Disallow an image directory to keep it out of image search
- Other engines use their own image-crawler tokens
Blocking pages vs blocking images
Blocking the HTML page that embeds an image does not necessarily remove the image from image search if the image file itself remains crawlable elsewhere. Conversely, blocking the image file's directory is the direct way to keep that image out of image results.
For a single image you do not want indexed, you can also use an X-Robots-Tag: noindex HTTP header on the image response — that requires the crawler to fetch the image, so do not also Disallow it in robots.txt, or the header is never seen.
How it appears in analytics and logs
Image-crawler hits (for example Googlebot-Image) on a directory mean that path's images are being fetched for image indexing. Blocking the path stops new image indexing of it.
Diagnostic use case
Keep specific image directories out of image search, or stop unnecessary image crawling, without accidentally removing images you want to appear in Google Images.
What WebmasterID can help detect
WebmasterID distinguishes image crawlers like Googlebot-Image from page crawlers, so you can see whether image crawling matches your robots.txt intent.
Common mistakes
- Disallowing an image directory and also expecting an X-Robots-Tag on it to be read — the header is never fetched.
- Blocking the page but leaving the image file crawlable, so it still appears in image search.
- Assuming one image-crawler token covers every search engine.
Privacy and accuracy notes
Image crawl rules concern your own files, not visitors. robots.txt is a request to compliant crawlers and does not secure private images — use authentication for that.
Related pages
- The max-image-preview robots directive explained
max-image-preview is a Google robots directive that bounds how large an image preview may appear for your pages in search results. This page explains its three values, where to set it, and why it matters for visual content and Discover-style surfaces.
- robots.txt for PDFs and non-HTML files
PDFs and other non-HTML files can rank in search. This page explains why X-Robots-Tag noindex (not robots.txt Disallow) is the right way to keep a PDF out of the index, and when blocking the file directory is appropriate.
- X-Robots-Tag header examples
X-Robots-Tag carries indexing directives in the HTTP response header instead of the HTML body, which makes it the way to apply noindex or nofollow to PDFs, images, and other non-HTML files. This page gives concrete header examples and notes how server config applies them in bulk.
- WebmasterID docs
How image and page crawlers are classified in your data.
Sources and verification notes
- Google — Overview of Google crawlers (Googlebot-Image)Documents the Googlebot-Image token used for image crawling.
- Google — robots meta tag and X-Robots-Tag (image noindex)X-Robots-Tag can apply noindex to non-HTML resources like images.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.