Cliff Sosin on the limits of intelligence: "Superintelligence arrived. You probably didn't notice, because it turned out to be kind of incremental. I think we overrated intelligence, or misread what it is. We hold a thin scatter of facts about the world, and intelligence or reasoning is whatever fills the space between them. Where the space between the facts behaves well, this is close to godlike. Most of what matters doesn't behave like that. The universe is mostly the emergent behavior of complex systems. Stir cream into coffee. Watch a storm build out of nothing but atmospheric temperature differences and water vapor. You have to run the thing. The takeoff story assumed the limit was thinking. It isn't. The limit is contact with reality. A smarter reasoner fills the gaps between known facts in simple areas faster and better, but it doesn’t produce new facts. An AI will hand you a genuinely clever design for a jet turbine blade. It might be far more likely to work than anything your engineers came up with. It'll still probably fail, because that’s the base rate at the edge of what anyone knows. The only way to find out is to build the blade and try to break it. That's the real limit on learning, and it doesn't care how smart you are. Coming up with ideas was never the hard part. The hard part is how fast reality answers them. Most problems can't be worked out at a desk by thinking. Those stay bound by how fast we can build the thing and see if it holds, and that clock barely moves." From his essay "Maybe Intelligence Ain't All That." He walks through it here.