Last week, a letter from mathematicians concerned about the effects of AI was dismissed as “protectionist”. I fear that what is happening to maths is a canary in the coalmine for a progressive hollowing out of intellectual institutions in the age of AI. Amidst discussions about AI’s potential to escape human control, a far more mundane but in my mind worse outcome gets slightly ignored: the danger that we might actually prefer to relinquish control to the machines, as human cognitive capacities atrophy. Optimistic accounts of an AI economy imagine machines doing cognitive work, while humans choose goals or exercise taste and agency. The problem I see is that the judgement, taste and purposes required for those roles develop through intellectual work and knowledge accumulation; they cannot be cleanly separated from these activities. So if we want humans to exercise wise control and ultimately, any control at all we have to preserve their cognitive abilities. But the intellectual communities through which these develops are threatened and I think the mathematicians’ letter is warning in that direction. For maths, the model seems to be following: If AI makes results cheap, the institutions supporting that training could lose their justification and funding, even as mathematical output multiplies. I worry that the maths community is the canary in the coalmine for a potential broader hollowing out of intellectual life. Human intellectual cultivation has historically been sustained indirectly by economic and social hierarchies, most recently through universities. In the most recent arrangement, access to desirable white-collar careers required passing through universities and society effectively subsidized institutions in which scholarship could flourish. This also had other positive spillovers: the preservation of a sense of a “unified civilization”, with a shared vocabulary of meaning, values and inherited traditions. I am by no means a supporter of universities in their current form and have criticized them at length (particularly because they have increasingly departed from the mission I outlined above). That being said, we do need some intellectual institutions. AI threatens the current order. If degrees become less necessary for employment, or if entry-level knowledge work falters, the economic incentives that support universities and other institutions of cultivation could weaken. This to me seems like one of the most important questions regarding “AI safety”. I know “cognitive decline” does not sound as scary or totalizing as existential risk, but the quality of human life matters just as much as the quantity of it. And to me, the value of technology is determined by its ability to elevate humans. I do not think that cognitive atrophy is pre-determined. I believe we can and should build institutions that preserve and encourage the cultivation of the human mind. But we have to take it seriously.