Yep crawler — YepBot (Ahrefs search)
Yep is a web search engine built by Ahrefs, and it is served by a crawler that self-identifies as YepBot. It is a genuine search-engine indexer, distinct from AhrefsBot (which powers Ahrefs' SEO/backlink datasets). Allowing YepBot can help your pages appear in Yep search results.
What this means
Yep is a web search engine developed by Ahrefs, the company best known for its SEO toolset. Yep's crawler, YepBot, fetches public pages to build the search index behind Yep results. Allowing it can help your pages appear in Yep.
Crucially, YepBot is distinct from AhrefsBot: AhrefsBot builds Ahrefs' backlink and SEO datasets, while YepBot supports the Yep search product. Operators who want search visibility but not SEO-data collection (or vice versa) should treat the two tokens separately.
How YepBot identifies itself
YepBot uses a YepBot token and includes a self-identifying URL in its user-agent string. Because Yep is a newer search product and its crawler is less exhaustively documented than the largest engines, this entry is marked partially verified — match on the YepBot token but confirm the current value and the AhrefsBot/YepBot distinction in Ahrefs/Yep documentation.
The user agent is a claim that can be copied; verify where authenticity matters.
- robots.txt token: YepBot (Yep search crawler; verify current value)
- User agent contains a Yep-identifying URL
- Distinct from AhrefsBot, which powers Ahrefs SEO datasets
robots.txt considerations
YepBot is expected to honour robots.txt like other search engines. To disallow it site-wide, target the YepBot token with a standard Disallow rule — note that this does not affect AhrefsBot, and blocking AhrefsBot does not affect YepBot.
Because YepBot is a search indexer, blocking it removes your pages from Yep results. robots.txt is a request honoured by compliant crawlers, not an access-control mechanism.
How it appears in analytics and logs
A request carrying the YepBot token is the Yep search engine fetching a URL for its index — a bot event, not a human visit. It is separate from AhrefsBot; YepBot relates to search results, while AhrefsBot relates to Ahrefs' SEO datasets.
Diagnostic use case
Confirm YepBot is indexing your site for the Yep search engine, keep it allowed for that visibility, and distinguish it from the AhrefsBot SEO crawler.
What WebmasterID can help detect
WebmasterID classifies YepBot server-side as a search bot and surfaces its activity on the bot-intelligence surface, separate from human analytics, so you can see Yep search-engine crawl coverage without log parsing.
Common mistakes
- Assuming one rule covers both YepBot and AhrefsBot — they are separate tokens.
- Blocking YepBot and unintentionally dropping out of Yep search results.
- Treating YepBot crawl hits as human traffic in analytics.
Privacy and accuracy notes
YepBot detection uses only the request user-agent. No human identity is involved. WebmasterID records the crawl as a bot event, separate from human analytics, and never attaches it to a visitor profile.
Related pages
- AhrefsBot — Ahrefs SEO crawler
AhrefsBot is the crawler operated by Ahrefs to build its SEO and backlink index. It is a third-party crawler, not a search engine, so it does not affect Google or Bing rankings directly. It uses the AhrefsBot robots.txt token and is documented as respecting robots.txt and crawl-delay.
- Brave Search crawler
Brave Search operates its own independent web index, and uses a dedicated crawler to gather pages for it. Brave documents the crawler and its robots.txt token. The exact token and verification specifics should be confirmed in Brave's documentation, so this entry is marked partially verified.
- Regional search engines overview
In several markets a regional search engine leads instead of Google: Yandex in Russian-language search, Baidu in China, Naver in South Korea, Seznam in the Czech Republic, and Coc Coc in Vietnam. Recognising their crawlers matters because being indexed by them is how you reach those audiences.
- Bot intelligence
Deterministic categorisation of search bots and SEO crawlers.
Sources and verification notes
- Yep — search engine by AhrefsYep is Ahrefs' search engine, served by YepBot; confirm current token and the AhrefsBot/YepBot distinction in docs.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.