AI crawlers · Mistral AI
Is your site blocking MistralAI-User?
MistralAI-User is the crawler Le Chat uses to open a page when a person asks it to. Check your site now: we read your robots.txt exactly as MistralAI-User must, ask your server as each AI crawler, and look at your product pages.
Does blocking it hurt?
Opens pages when a person asks Blocking MistralAI-User stops Le Chat from opening your pages when a person asks about them, so it answers without your own page.
In our research, 1% of 2,159 online shops block MistralAI-User in robots.txt. See the research.
Allow or block it in robots.txt
Mistral AI says robots.txt rules may not apply to MistralAI-User, because it acts for a person. A server or firewall block still stops it.
To let it in (add to your robots.txt, at yourshop.com/robots.txt):
User-agent: MistralAI-User Allow: /
To keep it out:
User-agent: MistralAI-User Disallow: /
A more specific group wins: if your robots.txt also has a group for MistralAI-User, that group decides, not the “User-agent: *” rules. Our check reads it the same way (RFC 9309).
Other places it gets blocked
- Cloudflare. Cloudflare has blocked AI crawlers by default on new sites since July 2025, and it can also write your robots.txt for you. If your site uses Cloudflare, check its AI crawler settings first (MIT Technology Review).
- Security plugins and firewalls. WordPress security plugins and host firewalls sometimes refuse crawlers they don’t recognise, even when robots.txt allows them. Our check asks your server as each crawler and shows any refusal.
Where this comes from
Mistral AI’s own documentation: docs.mistral.ai. Third-party directories: fetches pages cited in Le Chat answers; user-triggered. Confirm on Mistral's page.
We check 18 AI crawlers in one go. See them all · How the check works