How to block the OnPage.org / Ryte crawler
OnPage.org was an SEO site-audit platform that later became Ryte; its crawler fetches your pages to analyse technical SEO. This page shows the robots.txt token to target, notes the relationship to Ryte's crawler, and explains why a Disallow steers only compliant fetchers.
What the OnPage.org crawler is
OnPage.org was a technical-SEO audit platform whose crawler fetches a site's pages to evaluate on-page factors, structure, and crawlability. The company rebranded to Ryte, so you may see both the historical OnPage token and Ryte's crawler in logs.
Match on the documented OnPage user-agent token. If you also want to restrict Ryte's current crawler, target that token as well — see the dedicated Ryte crawler block page.
- robots.txt token: the OnPage.org crawler's self-identifying token
- OnPage.org rebranded to Ryte; both tokens may appear
- Purpose: technical-SEO site auditing
robots.txt rule
To ask the OnPage crawler to stay off your site, target its documented token with a site-wide Disallow:
User-agent: OnPage Disallow: /
Verify the exact token from the crawler's self-identifying user agent in your logs before relying on the rule. robots.txt is honoured by compliant crawlers and is not enforcement; if an uninvited audit continues, escalate to edge-level controls.
How it appears in analytics and logs
Requests carrying the OnPage crawler token are SEO-audit crawl events, not human visits. If you did not commission the audit, a competitor or third party may be analysing your site; classify the hits as bot traffic.
Diagnostic use case
Stop an SEO site-audit crawler from fetching your pages when the audit is not authorised by you, and confirm the rule reached the correct token.
What WebmasterID can help detect
WebmasterID classifies the OnPage/Ryte crawler server-side and shows whether it keeps reaching your pages after a robots.txt rule, so you can tell authorised audits from uninvited ones.
Common mistakes
- Blocking only OnPage and missing Ryte's current crawler token (or vice versa).
- Guessing the token instead of confirming it from the self-identifying user agent.
- Expecting robots.txt to enforce the block rather than request compliance.
Privacy and accuracy notes
Blocking the OnPage crawler uses only the request user-agent token. No visitor identity is involved, and WebmasterID records the crawl as a bot event separate from human analytics.
Related pages
- OnPage.org / Ryte audit crawler heritage
OnPage.org was a German SEO and website-quality platform that rebranded to Ryte. Its audit crawler fetched a customer's site to run technical and content checks. Older log entries may reference OnPage-era tokens. This entry documents the lineage and audit-crawler behaviour; exact historic UA strings are not asserted, so it is partially verified.
- How to block the Ryte crawler (botLogen)
Ryte is a technical-SEO platform (the rebrand of OnPage.org) whose crawler fetches pages to evaluate site quality and crawlability. This page shows the robots.txt token to target, notes the historical OnPage relationship, and explains why a Disallow steers only compliant fetchers.
- How to block SEOkicks
SEOkicks is a backlink-index service whose crawler fetches public pages to build a link database. This page shows the robots.txt token to target, explains what the crawler collects, and why a Disallow steers only compliant fetchers.
- Bot intelligence
Categorise SEO-audit crawlers reaching your site.
Sources and verification notes
- Ryte (formerly OnPage.org) — crawler informationOnPage.org rebranded to Ryte; token matched on the self-identifying user agent.
Last reviewed 2026-06-24. Facts are checked against primary/official sources where available; uncertain specifics are marked “Data not yet verified” rather than guessed.