PolicyOpenAIOctober 30, 2026

OpenAI will honor a shared robots.txt signal for AI training preferences

The title Making it easier for websites to express AI training preferences in white text over a yellow and green gradient. — Official imageImage: OpenAI (opens in a new tab)

robots.txt AI training signal at a glance

OpenAI says it will honor a common robots.txt signal, drafted in the IETF AI Preferences group, that tells several AI providers at once whether a site's content may be used for generative AI training. Implementation is planned in the coming months; existing crawler controls stay in place.

What changed with robots.txt AI training signal

On October 30, 2026, OpenAI announced that it will honor a common signal in robots.txt that lets website operators state whether their content may be used for foundational generative AI training. The signal comes from an IETF Internet-Draft linked to the AI Preferences working group, and OpenAI joined other participants in announcing it. Instead of writing separate rules for each provider's crawler, a site can express one preference that every participating provider reads.

As of November 5, 2026, the signal is not live at OpenAI: the company says it aims to implement it in the coming months and will publish implementation details then. The new signal does not replace OpenAI's existing robots.txt crawler controls, which continue to apply. OpenAI also urged other AI providers to honor the signal.

Why robots.txt AI training signal matters for businesses

For anyone who manages websites, this could make AI training opt-outs simpler, but it is still a draft and not yet enforced. Keep your current per-crawler rules, decide with each client whether to allow training separately from search and assistant visibility, and watch for OpenAI's implementation update before relying on the shared signal alone.

Sources for this story

Making it easier for websites to express AI training preferences (opens in a new tab)
OpenAI · openai.com
Official
Overview of OpenAI crawlers (opens in a new tab)
OpenAI · developers.openai.com
Official
draft-kelley-aipref-training-preference-signal (opens in a new tab)
IETF Datatracker · datatracker.ietf.org
Official
draft-ietf-aipref-vocab (opens in a new tab)
IETF Datatracker · datatracker.ietf.org
Official

Last verified: November 5, 2026. IA Compa summarizes public announcements for business readers. We are not affiliated with the companies mentioned. Always confirm details, pricing, and availability with the official source.

#openai
#robots-txt
#web-crawlers
#ai-training
#ietf
#seo

Oct 29, 2026OpenAIPolicy
OpenAI reorganizes its Model Spec around five values and drops most style rules
The October 29 Model Spec groups model behavior under faithfulness to human intent, harm minimization, honesty and transparency, human agency, and care. It adds guidance on context and side effects, retires most detailed style guidance and now calls the model GPT.
Image: OpenAI
Nov 5, 2026OpenAIModelsMajor
OpenAI introduces GPT-6 Cyber, a vetted-access security model rated Critical for cyber capability
GPT-6 Cyber, a security-tuned GPT-6 Astra variant, scores 48.5% on ExploitGym and found 81 high-severity Firefox flaws. As of November 5, 2026, only approved Daybreak Red customers can use it, via the API and Codex, at $20/$100 per million tokens.
Image: OpenAI
Nov 4, 2026OpenAISecurityMajor
OpenAI's autonomous pipeline built 948 Linux kernel root exploits, with patches, in one $1.2M run
OpenAI security researchers built an agent pipeline that finds, reproduces, exploits and patches Linux kernel bugs. One mid-October run produced 948 local privilege escalation exploits for $1.2M in API credits. No CVE IDs are named, and as of November 5, 2026, most fixes are still being reported.
Unresolved
Image: OpenAI