Goo search crawler — Japanese portal
Goo is a long-running Japanese web portal that offers search alongside dictionaries, news, and other services. Its search has historically relied on partner indexes, and a Goo-identified crawler may appear when fetching pages. This entry describes the documented pattern and is marked partially verified.
What this means
Goo is a Japanese web portal operated in the NTT ecosystem, offering search, dictionaries, news, and community services. Its search results have historically been powered in part by partner indexes rather than solely its own crawl, so search visibility on Goo can depend on the underlying index provider as well as any Goo-specific crawler.
For sites targeting Japanese users, Goo is one of several regional touchpoints alongside Yahoo! Japan and the global engines.
How a Goo crawler identifies itself
A Goo crawler self-identifies with a Goo token and a self-identifying URL in its user-agent string. Because Goo's search has relied on partner indexes and its own crawler is less exhaustively documented in English, this entry is marked partially verified — match on a Goo token if seen, but confirm the current crawler and index arrangement before treating it as authoritative.
The user agent is a claim that can be copied; verify where authenticity matters.
- robots.txt token: Goo's documented crawler token (verify current value)
- User agent contains a Goo-identifying URL
- A Japanese-market portal/search crawler, often index-partner backed
robots.txt considerations
A compliant search crawler honours robots.txt. To disallow a Goo crawler site-wide, target its confirmed token with a standard Disallow rule.
Because Goo search may draw on a partner index, your visibility there can depend on that provider as much as on a Goo-specific crawler, so blocking one token may not change Goo results as expected. robots.txt is a request honoured by compliant crawlers, not an access-control mechanism.
How it appears in analytics and logs
A request carrying a Goo crawler token is the Goo portal fetching a URL in connection with its search or portal services — a bot event, not a human visit. Its relevance is concentrated in the Japanese market.
Diagnostic use case
Recognise Goo-related crawler activity targeting the Japanese market and distinguish it from SEO tool crawlers and global search engines.
What WebmasterID can help detect
WebmasterID classifies a Goo crawler server-side as a search bot and surfaces its activity on the bot-intelligence surface, separate from human analytics, so you can see Japanese portal crawl activity without log parsing.
Common mistakes
- Assuming Goo runs a fully independent index without checking its partner arrangements.
- Confusing a Goo crawler with an SEO tool crawler.
- Treating crawler hits as human traffic in analytics.
Privacy and accuracy notes
Goo crawler detection uses only the request user-agent. No human identity is involved. WebmasterID records the crawl as a bot event, separate from human analytics, and never attaches it to a visitor profile.
Related pages
- Regional search engines overview
In several markets a regional search engine leads instead of Google: Yandex in Russian-language search, Baidu in China, Naver in South Korea, Seznam in the Czech Republic, and Coc Coc in Vietnam. Recognising their crawlers matters because being indexed by them is how you reach those audiences.
- Daum crawler — Korean portal (Kakao)
Daum is a major South Korean web portal, now operated under Kakao, offering search alongside news, mail, and community services. It runs a search crawler that self-identifies with a Daum token. It is a regional search-engine indexer, important for reaching Korean-market audiences.
- Yahoo Slurp — Yahoo's web crawler
Slurp is the web crawler historically operated by Yahoo. Modern Yahoo Search blends results from search partners rather than relying solely on its own index, so Slurp's role is narrower than it once was. Its robots.txt token is Slurp; current scope is best confirmed in Yahoo's documentation, so this entry is partially verified.
- Web crawlers overview
How WebmasterID separates regional search bots from SEO and AI crawlers.
Sources and verification notes
- Goo — Japanese portalGoo offers portal search, historically backed by partner indexes; confirm current crawler and index arrangement.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.