INDBInternet News Database 2.5 Updated just now · 292 sources Show or hide images Light or dark

Google and Microsoft among those signing up to OpenAI's plan for a new website crawling standard — including opting-out of internet scraping

Save Like Comments Compare coverage
Image from Techradar

Summary

One robots.txt line could soon tell Google, Microsoft, OpenAI, and Anthropic not to train on your site, but it won't stop their bots from crawling it.

Description from Techradar.

Who is covering this

  1. Techradar No political lean US
    Google and Microsoft among those signing up to OpenAI's plan for a new website crawling standard — including opting-out of internet scraping

    One robots.txt line could soon tell Google, Microsoft, OpenAI, and Anthropic not to train on your site, but it won't stop their bots from crawling it.

Comments

Comments need JavaScript.