Skip to content
Book a chat

For site owners

OtisBot

OtisBot is the web crawler operated by HeyOtis. It visits websites to measure how they are represented in AI search - whether a page is retrievable, what structured data it publishes, and whether AI assistants are able to reach it.

Last updated 18 August 2026

About OtisBot

We crawl on behalf of brands who ask us to analyse their own site, and we look at a small number of publicly available competitor pages to provide comparison. We do not collect personal data, we do not attempt to access anything behind a login, and we do not use what we fetch to train models.

How to identify us

OtisBot sends this user-agent string on every request:

OtisBot/1.0 (+https://heyotis.ai/bot)

We never disguise ourselves as a browser and never send another crawler’s name. If something calling itself OtisBot ignores your robots.txt, hammers your server, or tries to reach pages behind a login, it is not us — and we’d like to know about it.

Why we don’t publish IP ranges

Many crawlers publish a list of addresses they crawl from. We deliberately don’t, because ours would tell you nothing. We run on shared cloud infrastructure, so our outbound addresses belong to a pool used by thousands of unrelated services. Anyone could send you traffic from that same pool while calling themselves OtisBot.

Publishing it would also be an unreasonable thing to ask of you: allowlisting our range would mean allowlisting several hundred addresses we don’t control. We would rather give you no signal than a misleading one.

To control OtisBot today, use the user-agent in robots.txtbelow — we read it on every crawl. To verify a request really came from us, see Verification: every request we send carries a cryptographic signature, which proves identity in a way an IP list cannot.

How to control OtisBot

We read and obey robots.txt on every crawl, and we honour Crawl-delay.

To block OtisBot entirely:

User-agent: OtisBot
Disallow: /

To allow OtisBot but keep part of your site private:

User-agent: OtisBot
Disallow: /private/

Changes take effect on our next visit - we re-read robots.txt each time rather than caching it between crawls.

How we behave

  • Low volume. A typical analysis fetches a handful of pages per site: your homepage, robots.txt, sitemap.xml, llms.txt if present, and a small sample of product and content pages. We are not a bulk crawler.
  • Rate limited per domain, with an automatic backoff when a site returns 429.
  • We stop when asked. Repeated errors from a host end the crawl for that host rather than triggering retries.
  • Conditional requests where supported, so an unchanged page costs you almost nothing.

If you would rather we did not

Block us in robots.txt and we will stop - that is the whole mechanism, and we do not work around it.

If you’d prefer to talk to a person, email bot@heyotis.ai— a dedicated, monitored inbox for crawler enquiries — or use our contact page. We’ll answer.

Verification

OtisBot signs its requests under Web Bot Auth (RFC 9421 HTTP Message Signatures) — the emerging IETF standard for exactly this problem, already supported at the edge by Cloudflare and used by several major AI crawlers. Every request carries Signature, Signature-Input and Signature-Agent headers with the tag web-bot-auth, signed with an Ed25519 key.

Our public keys are published in a signed key directory at:

https://api.heyotis.ai/.well-known/http-message-signatures-directory

A request whose signature does not verify against those keys is not OtisBot, whatever its user-agent claims. Verification proves identity; robots.txt remains the control, and we honour it on every crawl.

See also our Privacy Policy and Terms of Use, or contact hello@heyotis.ai for anything else.