Company
OpenAI
Type
Training crawl
Obeys robots.txt
Yes
Announces itself as
GPTBot
Official documentation
https://platform.openai.com/docs/bots
What it does
OpenAI's primary crawler. Gathers publicly available web content used to train OpenAI's foundation models. Obeys robots.txt and publishes its IP ranges.
How Forge AI Bot Log checks it
Visits are checked against the IP ranges OpenAI publishes, so a spoofed GPTBot is labelled as such.
Should you block it?
Block it if you do not want your pages used to train OpenAI's models. Blocking GPTBot does not stop ChatGPT reading your pages live for a user, which is ChatGPT-User, or indexing them for ChatGPT search, which is OAI-SearchBot. Blocking GPTBot takes one switch in Forge AI Bot Log's Access Control screen. The plugin adds the rule to your robots.txt for you.