How to block Applebot in robots.txt
Applebot is Apple's crawler, used to power features such as Siri and Spotlight Suggestions. This page gives the robots.txt rule to disallow the Applebot token and explains that it is separate from Applebot-Extended, the token that governs AI-training use.
What Applebot is
Applebot is Apple's web crawler. Apple documents it as the crawler behind features such as Siri and Spotlight Suggestions. Blocking it can therefore reduce your visibility in Apple's search-style features, which may not be what you want.
Apple publishes a support article describing Applebot, its robots.txt handling, and how to verify it. Crucially, Applebot and Applebot-Extended are separate tokens: Applebot governs the crawl, while Applebot-Extended governs whether the fetched content may be used for AI training.
The rule
To disallow Applebot site-wide, target its token:
User-agent: Applebot Disallow: /
If your only concern is AI training, do not block Applebot — instead disallow Applebot-Extended, which leaves search features working while opting out of training use. Apple documents that distinction. robots.txt is a request to compliant crawlers, not enforcement.
- Token: Applebot
- Powers Siri and Spotlight Suggestions
- AI-training opt-out uses the separate Applebot-Extended token
How it appears in analytics and logs
A request carrying the Applebot token is Apple's crawler fetching a URL for its search features. Blocking it can reduce how your content appears in Siri and Spotlight Suggestions, so weigh that before disallowing.
Diagnostic use case
Disallow Applebot when you do not want your pages used in Apple's search features, while understanding that AI-training use is controlled by a different token.
What WebmasterID can help detect
WebmasterID classifies Applebot by its token as a search crawler, separate from human analytics, so you can confirm whether a disallow rule changed its activity.
Common mistakes
- Blocking Applebot when you only meant to opt out of AI training — use Applebot-Extended for that.
- Misspelling the token — it must be exactly Applebot.
- Assuming blocking Applebot has no effect on Siri or Spotlight visibility.
Privacy and accuracy notes
Blocking Applebot is a publishing-policy choice in a public file. It involves no visitor data and is not an access-control boundary.
Related pages
- Applebot — Apple's web crawler
Applebot is the web crawler operated by Apple, used to power features such as Siri and Spotlight Suggestions. It uses the Applebot robots.txt token and supports reverse-DNS verification. It is distinct from Applebot-Extended, a separate token governing use of crawled content for Apple's generative models.
- How to opt out of Apple AI with Applebot-Extended
Applebot-Extended is a robots.txt token Apple provides so site owners can opt out of having content used to train Apple's generative AI models. It is a standalone control: disallowing Applebot-Extended does not stop Applebot, which keeps crawling for Apple search features and Siri.
- How to block Yahoo Slurp in robots.txt
Slurp is Yahoo's historical web-crawler token. This page gives the robots.txt rule to disallow Slurp and explains that, because Yahoo Search has long been powered substantially by Bing, blocking Slurp may not change how Yahoo presents your site — verify what is actually crawling you first.
- Bot intelligence
See whether an Applebot disallow changed its activity.
Sources and verification notes
- Apple — About ApplebotDocuments the Applebot and Applebot-Extended tokens and robots.txt handling.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.