debate over whether AI-written or ghost-written op-eds gets to the aesthetics point I made in Tulsa and have been harping about on TMK and that others have made in one way or another. aesthetic and moral arguments against technology are pretty convincing and seductive in the moment, but sit at the lowest level of thinking about this technology. you have to articulate an opposition that will survive improvements in the technology. biometric surveillance opposition premised on racial bias was valid and made a lot of sense but the companies just went out and scanned faces of black & brown homeless to improve their capabilities and the argument fell to the wayside. what is the actual opposition? what is the opposition to what pervasive surveillance does to social relations and society at large? and what is the opposition to the proliferation of AI-generated text on a deeper level?

Consensus on this seems to be ZDR which tracks to my own experience as well. If that really turns out to be the case, it's an incredibly strong and clear message to the labs on the impact of data retention policies.

People just genuinely cannot fathom that those who work in AI are trying to be honest and candid about what the risks seem to be. I think it’s true that we undersell the benefits and are ~bad at marketing, but the notion that risk discourse is a branding tactic is simply wrong.

Sam Altman reveals the anti-human sales pitch that some people in the AI industry actually believe “It's a very misanthropic thing to say, we're going to just put all of our trust in this model and let it have all the power and decision making over the world. But there are some people who think that's the right outcome." “There's another version of this, which is because we don't trust people, we have to limit who gets access to this technology and how they can use it. And all of these terrible things could happen.” “Out of fear of that, we are going to concentrate power in the hands of a few companies. We're not going to let other people use this, but we'll give them some benefits." “I think there are some people in the AI field who effectively say, we're going to give the world a cure to all disease and make stuff really cheap in exchange for people giving up their autonomy and impact over the future and power.” “In the name of safety, and also just absolutely rampant inequality, there'll be people that have access to huge amounts of wealth and power and other people just get a pretty good everything." “I think it's fear and power. There are a lot of people who are so nervous about the magnitude of those risks and get so taken by that and feel a need to protect the world from that, that they're like, we should trade off a lot of liberty for safety here. But then I think that also ends up as a way a way to justify a lot of power seeking behavior."

A big reason Fable et al are annoying to talk to is that with long horizon coding training the vast majority of their conversational audience is themselves. They are literally being trained to talk better to themselves to do tasks, so it only makes sense that when you talk to them, they don't know how to talk to you. It's the opposite of RLHF.