OAI-SearchBot — OpenAI search crawler
OAI-SearchBot is the token OpenAI uses for crawling that supports its search features. OpenAI documents it as distinct from GPTBot, which crawls for model training, and from ChatGPT-User, the real-time browsing fetcher. It identifies itself with the OAI-SearchBot token plus a self-identifying URL.
What this means
OAI-SearchBot is the crawler OpenAI uses to support its search features. OpenAI documents it alongside GPTBot and ChatGPT-User, with each serving a different purpose: training, search, and real-time user browsing respectively.
Because these are separate tokens, a policy decision for one does not carry over to the others. If you want your content available in OpenAI's search experiences, allow OAI-SearchBot; if not, disallow that token specifically.
How OAI-SearchBot identifies itself
OAI-SearchBot uses the robots.txt user-agent token OAI-SearchBot. Its user-agent string contains that token together with a self-identifying URL pointing at OpenAI's bot documentation. Match on the stable token rather than a full version string.
The user agent is a claim and can be copied, so for requests that must be trusted, verify the source against OpenAI's published guidance rather than the user agent alone.
- robots.txt token: OAI-SearchBot
- User agent contains the OAI-SearchBot token plus an OpenAI URL
- Distinct from GPTBot (training) and ChatGPT-User (real-time fetch)
robots.txt considerations
OAI-SearchBot honours robots.txt. To disallow it site-wide:
User-agent: OAI-SearchBot Disallow: /
This affects only the search crawler. Blocking it does not block GPTBot or ChatGPT-User. robots.txt is a request to compliant crawlers, not an access-control boundary.
How it appears in analytics and logs
A request carrying the OAI-SearchBot token is OpenAI's search crawler fetching a URL — a bot event tied to search indexing, not model training and not a human visit. Treat it as crawl coverage for OpenAI search surfaces.
Diagnostic use case
Confirm whether OAI-SearchBot has crawled a page for OpenAI's search features and set robots.txt policy for it independently of GPTBot and ChatGPT-User.
What WebmasterID can help detect
WebmasterID classifies OAI-SearchBot server-side as an AI crawler and surfaces its activity on the bot-intelligence and AI-visibility surfaces, so you can distinguish OpenAI's search crawl from its training crawl and its real-time fetches.
Common mistakes
- Assuming one OpenAI rule covers GPTBot, ChatGPT-User, and OAI-SearchBot — each has its own token.
- Counting search-crawl hits as human traffic.
- Expecting robots.txt to enforce access rather than request compliance.
Privacy and accuracy notes
Detection uses only the request user-agent. No human identity is involved — a crawler is not a person. WebmasterID records the crawl as a bot event, separate from human analytics, and never attaches it to a visitor profile.
Related pages
- GPTBot — OpenAI's web crawler
GPTBot is the crawler OpenAI uses to fetch publicly available web content that may be used to help train its foundation models. It is a declared, well-documented crawler with a stable robots.txt token, and OpenAI publishes both documentation and an IP range list so operators can identify and control it.
- ChatGPT-User — OpenAI real-time fetcher
ChatGPT-User is the token OpenAI uses for real-time fetches made when a person in ChatGPT browses or asks it to read a URL. It is distinct from GPTBot, which crawls for model training, and OpenAI documents both. It honours robots.txt and identifies itself with the ChatGPT-User token plus a self-identifying URL.
- AI search analytics
Understand how AI search surfaces crawl and reference your site.
Sources and verification notes
- OpenAI — bots documentationDocuments the OAI-SearchBot, GPTBot, and ChatGPT-User tokens.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.