About our crawler

When someone checks a website with Known by AI, our crawler fetches that site’s home page, robots.txt, llms.txt and sitemap.xml. It identifies itself as KnownByAIBot/1.0, makes a handful of requests per check, and never fills in forms or logs in.

To stop it, add User-agent: KnownByAIBot and Disallow: / to your robots.txt, or email hello@knownbyai.co.uk and we’ll block checks of your site.