Yann LeCun, Executive Chairman of AMI Labs, explains why LLMs are mostly retrieving human knowledge rather than thinking for themselves:
LeCun starts with why so many people misread what these systems are doing:
"I think there's a lot of confusion, really, because we tend to anthropomorphize systems that can reproduce certain human functions."
When an AI writes fluent answers, we assume there's a mind behind them. LeCun's view is that most of what we're seeing is something more familiar:
"LLMs, to some extent, except for a few domains, are mostly information retrieval systems. They can compress a lot of factual knowledge that has been previously produced by humans and can give easy access to it."
The key words are "previously produced by humans."
The knowledge in an LLM came from people. What the system adds is compression and easy access.
That puts LLMs in a long historical line:
"In a way, it's kind of a natural evolution of the printing press, the libraries, the Internet, and search engines. Right. It's just a more efficient way to access information."
@ylecun is clear about their value:
"LLMs are incredibly useful, there's no question about that. And they do amplify human intelligence, like computer technology going back to the 1940s."
He also allows that in a few areas, such as generating code and some types of mathematics, the capabilities seem to go beyond retrieval. But he notes what those areas share:
"It's still, to a large extent, domains where reasoning has to do with manipulating symbols."
That's where the retrieval framing shows its limits. If these systems were truly thinking for themselves, you'd expect that ability to carry over into the physical world. It hasn't:
"The problem is that why do we have systems that can pass the bar exam and win mathematics Olympiads, but we don't have domestic robots, we don't even have self driving cars."
Then comes his sharpest comparison:
"And we certainly do not have self driving cars that can teach themselves to drive in 20 hours of practice like any 17 year old. So we're missing something big still."