IsVisible.ai

Overview

IsVisible.ai checks whether AI crawlers can actually reach a website. Paste in a URL and it tests against 13 crawlers and search bots: search engines (Googlebot, Bingbot), AI training crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot, Bytespider, Applebot-Extended, meta-externalagent), and AI answer engines (ChatGPT-User, OAI-SearchBot, Claude-User, PerplexityBot). It reads the site’s robots.txt and llms.txt, sends live requests using each crawler’s real user agent, and merges the declared rules with what actually happens to produce a 0-100 visibility score and an agent-by-agent access map.

Why it exists

The Hardal team has been building an AI visibility reporting product, one that shows companies which AI crawlers are actually visiting their site. Working on that, the same gap kept turning up: sites expected to show up when someone asked ChatGPT or Claude about them, with no idea they were blocking the exact crawlers that make that possible. Old robots.txt rules, a missing llms.txt, a WAF setting nobody had touched in years. IsVisible.ai is the free, standalone check for that specific gap.

Seeing it work

Run against this site, yigitalpkorpe.com scores 100 out of 100, Grade A: fully accessible to Google Search, Bing/Copilot, Google Gemini and AI Overviews training, GPTBot training, and the rest of the 13 crawlers it checks.

Under the hood

Getting the crawler simulation right took more tweaking than expected. Different bots honor different rules, and redirects, plus how a site responds to a user agent it’s never seen before, turned into edge cases that each needed their own handling. The tool is free with no signup, and also exposes an unauthenticated API for checking a site programmatically.