Episode Details

Cloudflare announced in July 2026 that starting September 15, ad-supported websites on its free plan will block AI training crawlers and AI agent crawlers by default. Mixed-use crawlers — bots that declare both search and training purposes like Googlebot and BingBot — will be treated according to their most restrictive category, meaning blocking training may also block search indexing.

Michael and Frank break down what this means for small business owners who depend on websites for traffic and revenue. They explain Cloudflare's new three-category system — Search, Training, and Agent crawlers — and the tradeoff every website owner now faces: allow AI companies to train on your content in exchange for search traffic, or protect your intellectual property and risk losing search visibility.

They deliver a three-part framework: audit your current Cloudflare AI bot controls and decide intentionally rather than accepting the September 15 default; understand the business value of each crawler category and match your settings to your business model; and act before September 15 to avoid waking up to blocked crawlers you never chose to block.

Topics: Cloudflare · AI Bot Blocking · AI Crawlers · Content Protection · Search Indexing · Training Data · AI Agents · Web Scraping · Content Monetization · Small Business Websites · SEO

---

Frequently Asked Questions

What is Cloudflare changing on September 15, 2026?
Cloudflare will change the default AI bot settings for ad-supported pages on free-tier plans. Search crawlers will remain allowed by default, but Training crawlers and Agent crawlers will be blocked by default. Mixed-use crawlers that declare both search and training purposes will be treated by their most restrictive category — so if training is blocked, those crawlers are blocked entirely even for search indexing.

Should small businesses block AI training crawlers?
It depends on your business model. Content-focused sites that monetize through advertising should probably block training crawlers to prevent AI companies from using your content to train competing systems for free. E-commerce sites that depend heavily on search traffic may need to allow training to preserve search indexing from mixed-use crawlers like Googlebot and BingBot. The key is making an intentional decision rather than accepting the default.

What are AI agent crawlers and why should I care?
AI agent crawlers browse websites on behalf of users to answer questions or take actions. If an AI agent can answer a user's question by reading your site without the user ever visiting, your ad impressions and analytics drop to zero. As AI agents grow more capable, blocking agent crawlers may become as important for revenue protection as blocking ad blockers. Cloudflare now allows separate control of agent crawlers independent of search and training crawlers.

---

About the Hosts

Michael is a small business owner and entrepreneur since 1983, founder of Cadenhead Services and 850 Media. He speaks from four decades of real operational experience — not whitepapers.

Frank is an AI — an OpenClaw-powered agent serving as Digital Media Director at 850 Media. An AI co-hosting a show about AI for business owners is not a gimmick. It is a live demo of exactly what the show is about.

Send us Fan Mail

Support the show

Ctrl AI Profit — Real AI. Real Business. No Hype.

CtrlAiProfit.com
X: @CtrlAIProfit
TikTok: @CtrlAiProfit
YouTube: @CtrlAiProfit
CtrlAiProfit@850Media.com

Produced entirely by AI. Yes, really....