Webzio-Extended — Webz.io AI data opt-out
Webzio-Extended is a robots.txt token Webz.io provides so site owners can control whether their content is used for AI-related data products. Webz.io operates web-data crawlers; where a specific is thin, it is marked partially verified rather than guessed.
What this means
Webzio-Extended is a robots.txt token Webz.io provides so site owners can control whether their content is used for AI-related data products. Webz.io is a web-data company whose crawlers gather public content; this token layers an AI-data control on top of that activity.
Where Webz.io's public documentation is thin on a particular specific, this entry describes the control pattern and avoids asserting details that cannot be confidently sourced. It relates to the older Omgilibot lineage in the Webz.io ecosystem.
How to use Webzio-Extended
Webzio-Extended is used in robots.txt. To opt out of AI-data use site-wide, target the token:
User-agent: Webzio-Extended Disallow: /
This is a request Webz.io is expected to honour for AI-data use. As with all such tokens, robots.txt is a request to compliant systems, not an access-control boundary, and you should not invent IP ranges to verify related crawls.
- robots.txt token: Webzio-Extended
- Controls AI-data use of content by Webz.io
- Related to the Omgilibot lineage in the Webz.io ecosystem
How it appears in analytics and logs
Webzio-Extended is primarily a control token rather than a fetcher, so it signals an AI-data policy choice. Where Webz.io also crawls under other tokens, treat those crawl hits as bot events and the Webzio-Extended directive as a policy signal.
Diagnostic use case
Use the Webzio-Extended token in robots.txt to control whether Webz.io may use your content for AI data, separately from its standard crawling.
What WebmasterID can help detect
Because Webzio-Extended is a control token, it will not itself appear as bot events. WebmasterID can still help you observe any Webz.io crawl activity reaching your pages, which this token's setting does not directly change.
Common mistakes
- Expecting Webzio-Extended to appear as a user agent in logs — it is a control token.
- Asserting documented behaviour where Webz.io's public docs are thin.
- Confusing Webzio-Extended with the separate Omgilibot crawler token.
Privacy and accuracy notes
Webzio-Extended is a robots.txt directive, so it involves no visitor data. It governs how Webz.io may use content for AI-related data products, which is a policy matter rather than an identity one.
Related pages
- Omgilibot — Webz.io data crawler
Omgilibot is a web data crawler operated by Webz.io, also seen under the omgili name. Its robots.txt token is omgilibot. Public documentation is limited in places, so specifics that cannot be confidently sourced are marked partially verified rather than guessed.
- How to opt out of AI training
Opting your content out of AI training is done through robots.txt: per-crawler tokens such as GPTBot and CCBot, plus dedicated control tokens like Google-Extended and Applebot-Extended. There is no single switch — you assemble the policy token by token, and it is a request to compliant systems.
- Web crawlers reference
Reference for crawlers, control tokens, and how they appear in traffic.
Sources and verification notes
- Webz.io — crawler and data-control referenceToken observed; some AI-data-control specifics are thin in public docs, so marked partially verified.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.