This article is based on a single primary source and has not been independently corroborated. DeepBrief is monitoring for additional confirmation. Cloudflare announced a feature called Redirects for AI Training that converts existing <link rel="canonical"> tags into HTTP 301 redirects for verified AI training crawlers, according to a post published on the Cloudflare Blog on April 17, 2026. The company says the feature is available on all paid Cloudflare plans as a one-toggle setting and targets a problem Cloudflare describes as crawlers ingesting deprecated content despite advisory signals.

Source: https://blog.cloudflare.com/ai-redirects/

The data Cloudflare cites on crawler behavior

Cloudflare reports that bots in its AI Crawler category visited developers.cloudflare.com 4.8 million times over the last 30 days, based on telemetry from its AI Crawl Control product. The company says those crawlers "consumed deprecated content at the same rate as current content" and that advisory signals such as noindex meta tags and canonical tags "made no measurable difference."

Cloudflare frames the issue as cumulative. "When crawlers ingest deprecated docs, agents inherit outdated foundations," the post states, noting that AI agents often draw on trained models rather than fetching content live.

The company also published figures for specific crawlers. In March 2026, according to Cloudflare, legacy Workers documentation was crawled roughly 46,000 times by OpenAI, 3,600 times by Anthropic, and 1,700 times by Meta. DeepBrief has not independently verified these counts.

How the redirect mechanism works

According to Cloudflare's documentation, the feature operates on two inputs: the cf.verified_bot_category field used by Cloudflare's ruleset engine, and the <link rel="canonical"> tags already present in a site's HTML. The AI Crawler category, Cloudflare says, covers bots including GPTBot, ClaudeBot, and Bytespider, and is distinct from the separate AI Assistant and AI Search categories that cover AI agents.

When a request arrives from a verified AI crawler, Cloudflare says it reads the response HTML, and if a non-self-referencing canonical tag is present, issues a 301 Moved Permanently to the canonical URL before returning the response. The company states that human traffic, search indexing, and other automated traffic are not affected.

Your existing canonical tags become HTTP 301 redirects for verified AI training crawlers, automatically, with one toggle, on all paid Cloudflare plans.

Cloudflare published an example exchange in which a GPTBot request to a deprecated Durable Objects path receives a 301 redirect to the current SQLite storage API documentation.

Scope and exclusions

Cloudflare lists several limits on what the feature does. According to the post, it does not retroactively correct training data already ingested, and it does not cover unverified crawlers outside the AI Crawler bot category. Humans and AI agents visiting deprecated pages are not redirected.

The company says it excludes cross-origin canonicals by design, because tags directing to preferred URLs on different domains are "often used for domain consolidation rather than content freshness." Self-referencing canonical tags also do not trigger a redirect, which Cloudflare states is intended to avoid loops.

Cloudflare notes that the <link rel="canonical"> tag is defined in RFC 6596 and cites the 2025 Web Almanac figure that canonical tags are present on 65–69% of web pages, generated automatically by platforms including EmDash, WordPress, and Contentful.

Why Cloudflare says redirect rules are not sufficient

Cloudflare addressed why developers might not simply use its existing Single Redirect Rules to target AI crawlers by user-agent string. The company says that approach "doesn't scale: every new deprecated path requires a change to the rule, user-agents must be manually tracked, and it would contribute to plan limitations that may otherwise be used for campaign URLs or domain migrations." The post adds that redirect rules "manually re-encode what canonical tags already declare and fall out of sync as content changes."

The company also describes robots.txt as offering limited protection, arguing that per-crawler, per-path directives require manual upkeep as automated traffic grows.

A reported example from Cloudflare's own documentation

Cloudflare includes an anecdote from its own testing. The company says that in April 2026 it asked an AI assistant how to write KV values using the Wrangler CLI and received the response: "You write to Cloudflare KV via the Wrangler CLI using the kv:key put command."

According to Cloudflare, the correct syntax as of April 2026 is wrangler kv key put, and the colon syntax was deprecated in Wrangler 3.60.0. The company says its documentation carries an inline deprecation notice but that "it's unclear how training pipelines interpret them." DeepBrief was not able to independently reproduce this exchange.

Associated Radar update

Alongside the feature launch, Cloudflare said its Radar AI Insights page now includes a Response status code analysis view showing the distribution of 2xx, 3xx, 4xx, and 5xx responses that AI crawlers receive across Cloudflare traffic. The company describes it as "a view of how the web responds to AI crawlers today."

Cloudflare's post does not disclose third-party compliance data from the AI companies whose crawlers it names, and DeepBrief has not located independent measurements of canonical-tag handling by GPTBot, ClaudeBot, or Bytespider at the time of publication.