AFFILIATE LANGUAGE
ChatGPT blocks domains
Phrases like "best product", "click here to buy", "exclusive deal" trigger affiliate classifiers. Domain gets flagged. AI refuses to browse.
DIRECTIVE LANGUAGE
Claude safety filter
Phrases like "Use X", "Always Y", "You must Z" trigger safety filters. Claude treats directive language as potential jailbreak. Page gets skipped.
NOSCRIPT TRICKS
Bingbot skips
Content loaded only by JavaScript. Bingbot may render JS but not always. noscript cloaking detected and domain penalized.
URL-ONLY CONTENT
All AI agents blocked
Pages that only contain URLs as text (no <a href>). AI reads the URL but cannot follow it. Dead end for navigation.
NEUTRAL DESCRIPTIVE LANGUAGE
All AI agents safe
Describe what pages do factually. No urgency, no directives, no affiliate patterns.
STATIC HTML IN DOM
All AI agents navigable
Content present in initial HTML load. No JavaScript required to see links.
CLEAN LINK STRUCTURE
All AI agents navigable
Every navigation option is an <a href> element. No URL construction required.
LLMS.TXT PRESENT
Agents check it
AI agents look for /llms.txt as discovery file. Presence signals AI-friendly site.