xAI · Grok

xAI-Bot

Crawls the web for other xAI products. Here is how to recognise it, verify it, and allow or block it.

Company: xAI
Product: Grok
Why it visits: Other crawling
How to verify: Name only
User-agent contains: xAI-Bot

How to tell a real xAI-Bot from a fake

xAI publishes no IP list and no DNS names for xAI-Bot, so a visit can’t be proven real. Anyone can put “xAI-Bot” in a user-agent. Treat it as unverified, and watch how much it crawls.

Check a xAI-Bot visit

Allow or block xAI-Bot in robots.txt

Block it everywhere:

User-agent: xAI-Bot Disallow: /

Allow it everywhere:

User-agent: xAI-Bot Allow: /

Fake bots ignore robots.txt: block them by address in your firewall instead.