99.7% of Websites Are Invisible to AI Search

A comprehensive analysis of one million websites has revealed a startling gap in the digital landscape: only 0.3% have implemented AI discovery files, leaving the vast majority effectively invisible to the artificial intelligence systems now answering billions of queries daily.

The finding comes as ChatGPT processes 2.5 billion prompts each day from its 800 million weekly active users, whilst Google’s AI-powered search generates answers without requiring users to click through to websites in 58% of searches.

“If your content isn’t in the Docs section of an llms.txt file, your brand and your narrative will not be found,” warns Profound, a company specialising in generative engine optimisation metrics.

The Adoption Crisis Leaving Businesses Behind

Research conducted by Frank Sauerburger analysing the top one million domains found AI discovery files adoption rates declining sharply as website rankings decrease. The data reveals a troubling pattern:

Rankability’s monthly monitoring of the top 1,000 most-visited websites found that “not a single major platform has implemented llms.txt transparency standards” among sites like Google, Facebook, and Amazon.

What AI Actually Sees Without AI Discovery Files

The implications extend beyond mere statistics. When artificial intelligence systems encounter websites without proper discovery files, they face significant technical barriers.

“Large language models increasingly rely on website information, but face a critical limitation: context windows are too small to handle most websites in their entirety,” explains Hungry Ram Web Design in their analysis. “Converting complex HTML pages with navigation, ads, and JavaScript into LLM-friendly plain text is both difficult and imprecise.”

Without structured guidance, AI systems must parse entire websites—complete with styling, scripts, and markup—consuming excessive computational resources whilst potentially misinterpreting core business offerings.

The Early-Mover Advantage Window

For UK businesses, the low adoption rate represents an unusual opportunity. Companies implementing AI discovery files today position themselves amongst the 0.3% of websites that AI systems can efficiently understand and recommend.

365i’s free AI Visibility Checker allows businesses to assess their current standing within seconds, providing a score and specific recommendations for improvement.

“Today, being accessible to LLMs gives you a competitive advantage. Soon, it will become table stakes,” notes Mintify in their assessment of the evolving landscape.

The Implementation Gap

Despite the clear opportunity, significant barriers to adoption remain. The technical requirements, whilst not insurmountable, require specific expertise.

AI discovery files encompass multiple components: llms.txt for core business information, ai.txt for operational parameters, brand.txt for identity consistency, faq-ai.txt for common queries, developer-ai.txt for technical documentation, robots-ai.txt for crawler permissions, identity.json for structured data, and llms.html for visual representation.

365i’s professional AI Site Identity service addresses this complexity, implementing all eight discovery files with proper structure and formatting for £295 plus VAT, completed within 24-48 hours.

The Cost of Inaction

As AI-powered search continues its rapid expansion, businesses without proper discovery files face mounting disadvantages. Research from Fullview indicates ChatGPT doubled from 400 million users in February 2025 to 800 million by September—a six-month period during which most websites remained invisible to these systems.

The competitive gap widens daily. Whilst 78% of enterprises now use AI in at least one business function according to Netguru’s analysis, the disconnect between internal AI adoption and external AI visibility remains stark.

For WordPress websites—which power 43% of the internet—the implementation challenge presents both obstacle and opportunity. Most WordPress users focus on traditional SEO whilst artificial intelligence systems processing their site struggle with plugin-heavy codebases and complex page structures.

What Success Actually Requires

Effective AI discovery implementation demands more than simply creating text files. The content must be structured precisely, using markdown formatting that AI systems expect, with clear hierarchical organisation and specific business information.

Each file serves distinct purposes: llms.txt provides the entry point and navigation, directing AI to relevant information sections. The developer-ai.txt file contains technical specifications for software-oriented businesses. The faq-ai.txt addresses common customer queries in formats optimised for AI comprehension.

Businesses attempting DIY implementation without technical expertise risk creating files that either confuse AI systems or, worse, provide incorrect information that becomes embedded in AI training data.

Measuring the Impact

Unlike traditional SEO where metrics are well-established, AI visibility measurement remains emerging territory. Businesses can track several indicators: AI system citation frequency, referral traffic from ChatGPT and similar platforms, accuracy of AI-generated business descriptions, and appearance in AI-powered search results.

Position Digital’s research found that “sites with over 32,000 referring domains are 3.5 times more likely to be cited by ChatGPT than those with up to 200 referring domains,” suggesting that whilst discovery files matter, they function alongside established authority signals.

The Broader AI Visibility Context

AI discovery files represent one element of a comprehensive AI visibility strategy. Research by SE Ranking examining 300,000 domains found that “domains with profiles on platforms like Trustpilot, G2, Capterra, Sitejabber, and Yelp have three times higher chances to be chosen by ChatGPT as a source.”

The data suggests AI systems prioritise established brands with strong digital footprints. However, this makes proper discovery files even more critical for smaller businesses seeking to compete against larger, better-resourced competitors.