Crawler visibility diagnostics
People ask ChatGPT what to buy long before they reach you. Agentic Lens requests your pages the way those crawlers do and shows you exactly what came back. If your prices aren't getting through, you'll see the proof rather than someone's opinion.
We only fetch pages any crawler can already reach, and we never write anything to your site.
Five segments read. One arrives empty.
A real result
One store, one product page, seven crawler identities. Under the output we translate the three lines that matter, so you don't need to read a terminal to know where you stand.
gptbot 403OpenAI's crawler is turned away at the network. It's never loaded this product page.
price absent from raw HTMLShoppers see the price. The model answering questions about your category never has.
claudebot inconclusiveWe couldn't prove this one from outside, so we leave it unscored and tell you where to check.
Two of those lines are costing this store visibility today. The third is one we won't guess at, because a wrong accusation is worth less to you than an honest gap.
See yours in about 40 secondsWhy this happens
Modern storefronts build the page inside the browser. Your customer's browser runs that code, so the price appears. Most AI crawlers don't run it, so they get the page without the price. Nothing errors, no traffic drops, nobody files a ticket. It's invisible from the inside, which is why almost nobody has checked.
of the storefronts we scanned put price and availability in the page as delivered, not just after the browser builds it
First 100 stores, full dataset in progress
answer an AI crawler differently than they answer an ordinary browser, usually without anyone deciding to
First 100 stores, full dataset in progress
of checks can't be verified from outside. We report those as unproven instead of calling them a block
Our own telemetry since launch
More content won't fix this, and neither will more ads. A price that isn't in the response was never read.
Check your own pagesWhy you can trust the answer
Content delivery networks recognise real crawlers by IP address, not by name. A request that only claims to be GPTBot can get challenged even when the genuine GPTBot is welcome. That's why we run an ordinary browser and our own honest bot alongside it, and why a proven block never looks like an unproven one here.
| Code | Requested as | Status | Body hash | What we conclude |
|---|---|---|---|---|
| REF-BROWSER | Reference browser | 200 | 4f09ab | Baseline for comparison |
| NEUTRAL-CTL | AgenticLensBot | 200 | 4f09ab | Allowed through |
| GPTBOT | GPTBot | 403 | none | Blocked, and we can prove it |
| CLAUDEBOT | ClaudeBot | 403 | cf-chl | Challenged, block unproven |
| OAI-SEARCH | OAI-SearchBot | 200 | 91c7d2 | Served different content |
| PERPLEXITY | PerplexityBot | 402 | none | Pay per crawl, your choice |
| CHATGPT-USER | ChatGPT-User | 503 | none | Site was down, nothing scored |
Every row carries its status code, its body hash and the moment it was captured, so your engineers can reproduce it. Where we can't prove something, we say so and leave it out of the score.
Run this on your siteWho it's for
The scan is free for anyone, and you're welcome to run it whatever you sell. The paid work is narrower, and it's worth being blunt about where it pays for itself.
How it works
The free scan and the paid audit run the identical sequence. The audit goes further. It isn't done more carefully.
Your home page, a product page and your pricing page, each requested seven times: once as an ordinary browser, once as our own honest bot, and five times as the AI crawlers that matter. We record the status, the headers and the body every time.
Each answer is measured against the ordinary browser and against our honest bot. That separates a rule aimed at AI crawlers from one aimed at every bot, and a genuine block from a challenge a verified crawler would pass.
Findings land in four groups, and two of them mean we don't know. If the ordinary browser was blocked too, your site is refusing everyone and we make no claim about AI at all.
A permanent link with the table, the evidence and a list of everything that passed. If we find nothing wrong, that's a documented verification you can forward, not a wasted scan.
Before you paste your address
No. We request a handful of public pages, the same ones any crawler can already fetch. We never log in, never submit a form and never write anything. The whole scan is a few dozen requests.
Not to run it. You paste your address and read the result. You'll want them for the fixes, which is exactly why the result page is built to be forwarded with the evidence attached.
No, and be careful with anyone who does. Nobody controls what a model decides to cite. What we can prove is whether your content reaches the crawler at all, and that part you do own.
A readability index built from the checks we ran. We publish it with its version and its method, and we label it uncalibrated because we haven't yet shown it predicts citations. When we can show that, we'll say so.
Your result page isn't indexed by search engines, and we don't name domains in anything we publish. The sector report shows aggregates only. Your own findings go to you.
The deep audit is $290 once, with no subscription. It adds the transcript of a model working only from what your crawler received. If you then hire us to fix things, that $290 comes off the price in full.
Find out now
Paste your address and Agentic Lens hands back the full diagnosis with the evidence attached, on a page you can send straight to your team. No account, no card, no call.