CeyoBot

CeyoBot is the web crawler of Ceyo, an AI search visibility platform. This page explains what it does, how to recognise it and how to control its access to your website.

What CeyoBot does

CeyoBot visits websites on behalf of Ceyo customers. It:

• audits a customer's own website for search and AI visibility, when they ask for it;
• reads public pages that AI assistants cite in their answers, to understand what they say about a business;
• checks that a website or domain added to Ceyo exists and responds.

How to recognise CeyoBot

Every request from CeyoBot sends this user agent:

Mozilla/5.0 (compatible; CeyoBot/1.0; +https://www.ceyo.ai/bot)

In robots.txt, CeyoBot follows the rules for this token:

CeyoBot

What it fetches

CeyoBot only reads publicly available pages and files, such as web pages, robots.txt, sitemaps and llms.txt. It does not log in, fill in forms or place orders, and it does not collect personal data.

What CeyoBot fetches is used to produce reports for Ceyo customers. It is not used to train AI models.

Crawl rate

CeyoBot checks a limited number of pages per visit and spaces its requests out, so it adds very little load to your server.

Controlling access with robots.txt

CeyoBot reads robots.txt and skips pages that are disallowed for it. To keep CeyoBot out of your whole website, add:

User-agent: CeyoBot
Disallow: /

One exception: when a website's owner, or someone acting for them, asks Ceyo to audit that website, CeyoBot checks the pages selected for the audit even if robots.txt disallows them. Part of the audit is reporting whether your robots.txt blocks search engines and AI crawlers.

Allowing CeyoBot through a firewall

If you use Ceyo to audit your website and a firewall or bot protection service blocks the audit, create an allow rule for the user agent above. When an audit is blocked, your Ceyo account shows the IP addresses to allow.

Questions

Questions about CeyoBot, or think it is causing problems on your website? Email us: support@ceyo.ai