Episode Details

E1123: Claude shared conversations were appearing in Google, Bing, and other search engines, exposing information that users may never have expected to become publicly searchable.

The problem was not caused by a complicated search engine failure. It came down to a basic technical SEO mistake: the shared pages were accessible to search engines without the correct indexing controls in place.

In this episode, I break down what happened, why the reporting around the story created additional confusion, and what the incident shows about the importance of involving experienced SEOs in product and website decisions.

Topics covered: - How Claude shared chat URLs became publicly searchable - The types of sensitive information reportedly found in indexed conversations - Why a public URL can be discovered even when users think it is only being shared with one person - Why robots.txt does not guarantee that a URL will stay out of search results - How a noindex directive works - Why blocking a page in robots.txt can prevent search engines from seeing its noindex directive - The mistake Anthropic made when handling Claude shared pages - The incorrect SEO advice included in some reporting about the incident - What Google's documentation says about robots.txt and noindex - Why search engines are not responsible for deciding which website pages should remain private - How website owners can control which pages are crawled, indexed, and displayed in search results - Why sensitive pages should be protected at the product and technical level - How ChatGPT shared conversations created a similar indexing problem in 2025 - How SEOs used indexed AI-generated pages for parasite SEO - Why some old Claude Artifact pages may still appear in Google - What this incident reveals about the continued importance of technical SEO

A key distinction discussed in the episode is the difference between crawling and indexing.

A robots.txt rule can prevent a search engine from crawling a page, but it does not necessarily prevent the page's URL from appearing in search results. If Google discovers the URL through links or other sources, it may still index the URL without being able to read the page.

A noindex directive tells a search engine not to include a page in its index. However, the crawler must be able to access the page to see that directive. When a page is blocked through robots.txt, the crawler may never see the noindex instruction.

This is why combining a robots.txt block with an on-page noindex directive can create problems. The crawler is blocked before it can read the instruction telling it not to index the page.

The broader lesson is straightforward: - Decide which pages should be public - Decide which pages should appear in search engines - Do not assume that an unlisted URL is private - Use the correct technical controls - Test how search engines can access sensitive page types - Include SEO specialists when building sharing, publishing, and privacy features - Remove sensitive information at the source instead of relying only on search engine directives

SEO is not only about increasing rankings or generating traffic. It also includes controlling what search engines can discover, crawl, index, and display.

When those controls are handled incorrectly, the result can be a privacy problem, a security problem, and a reputational problem.

⭐️ Om Patel's Claude post - https://x.com/om_patel5/status/2081218866839966116 ⭐️ What excites you in SEO - https://x.com/edwardeachday/status/2083151456685769104 ⭐️ Barry Schwartz writeup - https://searchengineland.com/google-indexed-claude-chats-because-anthropic-didnt