Skip to content
MaungaAI news, explained

Society

Four major AI companies say they will adopt a signal that lets websites tell them no

OpenAI says it will respect a common signal and, according to The Verge, Google, Microsoft and Anthropic will adopt it too. Websites could decide all at once, but it's still unclear exactly what it covers.

2 min read2 sources, one officialTranslated with AI from the Spanish original

Share: WhatsApp · Email

AI-generated illustration for Maunga. Not a photograph.

The essentials

  • Claimed by one partyOpenAI says it will respect a common signal in robots.txt, the file that shows which programs can crawl a website (OpenAI News, The Verge).
  • Claimed by one partyAccording to The Verge, Google, Microsoft and Anthropic also say they will adopt that standard (The Verge).
  • Claimed by one partyAccording to The Verge, a website could apply its preferences to several AI companies at once, instead of one by one (The Verge).
  • Claimed by one partyAccording to The Verge, the proposal comes from Kevin Kelley, technical standards lead at Anthropic, within the Internet Engineering Task Force (IETF) (The Verge).

Why it matters

Less work for anyone who publishes online: according to The Verge, preferences are currently set company by company (The Verge). If this goes ahead, a single signal would be enough for several companies. How far it reaches will depend on who takes part, because OpenAI talks about participating providers, not the whole industry (OpenAI News).

What does it mean for me?

If you have a website or a blog, you could state your preferences to several AI companies at once. If you only browse the web, it shouldn't directly affect you, in principle.

The details

The Verge calls this signal a "policy token", a kind of permissions label inside robots.txt. For years, that file has shown which crawlers can collect a website, whether for search engines or for archiving it (The Verge). The outlet adds that Search Engine Roundtable reported it first (The Verge).

OpenAI talks only about training preferences and about the companies that take part (OpenAI News). The Verge talks about those preferences, but also about opting out of crawling (The Verge). That's not quite the same thing: not using a website to train an AI is one thing, and not collecting its content is another.

What we don't know

  • When the four companies will start applying the signal.
  • Whether it covers only training or other uses of crawling as well: The Verge talks about both and OpenAI only about training.
  • Which other companies will take part, besides the four mentioned.

Sources

This story is based on 2 sources. In the text, each fact names in brackets who reports it.

Was this story useful? Yes Not really

Spanish original