Cohere · Cohere

cohere-training-data-crawler

Collects pages to train Cohere’s AI models. Here is how to recognise it, verify it, and allow or block it.

Company: Cohere
Product: Cohere
Why it visits: Collecting for AI training
How to verify: Name only
User-agent contains: cohere-training-data-crawler

How to tell a real cohere-training-data-crawler from a fake

Cohere publishes no IP list and no DNS names for cohere-training-data-crawler, so a visit can’t be proven real. Anyone can put “cohere-training-data-crawler” in a user-agent. Treat it as unverified, and watch how much it crawls.

Check a cohere-training-data-crawler visit

Allow or block cohere-training-data-crawler in robots.txt

Block it everywhere:

User-agent: cohere-training-data-crawler Disallow: /

Allow it everywhere:

User-agent: cohere-training-data-crawler Allow: /

Fake bots ignore robots.txt: block them by address in your firewall instead.