chatgpt is now indexing the web
they're indexing the web like google
researchers found its name in ChatGPT's traffic called Labrador
from May 21 to July 21, every search result in ChatGPT carried a label for where it came from
three of the labels were scraping services that pull Google results. the fourth was Labrador, which is OpenAI's own index
a few things stood out to me
it barely overlaps with Bing. Researchers ran the same searches on both, and only about 1.5% of Labrador's results were in Bing's top 20
if you've been treating your Bing rankings as your ChatGPT rankings, they're not the same thing.
free users mostly get Labrador
in the default Instant mode, ChatGPT didn't open a single page in 93% of answers
it answered from a stored snippet of about 200 characters
Paid Thinking mode is different
about 75% of its results came from scraped Google, and it actually reads the pages
your snippet comes from your H1 and the text right around it
ChatGPT ignores your meta description, and the snippet doesn't change until the page gets re-indexed
updating a page doesn't update ChatGPT. it keeps a cached copy of your page. pages nobody asks about can stay stale for months. it also doesn't run JavaScript, so anything that loads with JS isn't there
OpenAI runs three separate bots. GPTBot collects training data. OAI-SearchBot decides whether you show up in ChatGPT search. ChatGPT-User fetches a page live during a chat. I keep seeing sites that blocked all three to stay out of training and then wonder why ChatGPT never cites them
so what I'd do:
allow OAI-SearchBot in robots.txt.
make sure your H1 still makes sense if someone reads only that line
answer the question in your first paragraph
put the date and author in visible text near the top
render your content on the server
ChatGPT still mixes in Google and Bing results. But OpenAI is already testing "prefer our own index" in shopping, so I'd expect more answers to come from Labrador over time
Research credit: Peec AI, Search Engine Land, RESONEO, a lot of X posts