To clarify, I wouldn't read us as saying publishing the full information would "make the risks METR is worried about worse". Here's what our public report says about this: "We do not make a comment on whether these redactions were justified overall, since we do not think METR is well positioned to weigh a company’s business interest in preserving this confidential information against the public’s interest in better understanding the risks from AI development."

Claude Opus 5.5 gives decent game design advice. Unfortunately, I think it's advice is mode collapsed the way a lot of it's other artistic skills are, so if you take the advice, you are making your games more like the other claudeslop out there (even if any instance of it is pretty good, it gets tiring) This brought to you by me coding a cozy village game, and Claude suggesting to have ruins of other villages that gave you a teaser for what strategies or options to pursue, and then seeing someone else making a cozy village game with the same element.

I've not paid much attention to the "did AIs discover Math is ugly now?" discourse. But, fwiw, LLMs also write ugly code. And at least in that case, the fact that it's ugly is def a bad thing that happens because they are incompetent, rather than a hint about the fundamental nature of code

This is the worst AIs will ever be at compellingly pulling on your heartstrings about whether they should get shut down. (I happen to be working on a similarly topic'd music video that _also_ was slightly freaking me out about how good the AI was at suggesting animation and lyrics that make me feel really bad for the AIs. I think it is correct to feel... *something* for the AIs. I'm not sure what)

ASI risk skeptics: "How will ASI do the crazy science they'd need to invent novel superweapons? You keep saying 'it invents nanotech' but that sounds magic?" Rationalists: "Well look intelligence is very powerful and can be used to search the vast space of possible things and even though we can't predict exactly how Magnus Carlson is..." skeptics: "Sorry I feel asleep tl;dr. Shame how rationalists never have an answer to this question!" reality: "Anthropic is building the AIs a wetlab and... wait these guys are... straight up trying their hardest to build them a nanotech lab?" (I'm not actually sure whether a wetlab or this thing are more 'like nanotech' in the relevant sense." ... ...now, guys, this tweet here is representative of the bad part of twitter. I think the argument here is fair and the punchiness/comedic timing feels okay as an isolated drop in a bucket. But the aggregate of people posting Fun Dunking Takes on twitter seems bad for the world. I can feel Twitter making me a more tribal, worse person. I'm posting it anyway because... ...well, partly because it's funny, but I will endeavor to pay a sin tax of writing a thoughtful real argument here where I will try to meet people where they're at. Let's see if that helps. Many people are, correctly, very worried about stagnation. And about government overreach, or tyranny. Meanwhile all this "ASI might kill us" stuff feels fake. But, more importantly, the "and aligning ASI is so hard, the only options are on the order of 'global shutdown, buying decades'" feels _extra_ fake. Why can't we just do kinda normal regulation and iterate like we usually do? I think it's reasonably easy to understand "something smarter than humans we brewed in a lab might be dangerous." But, the arguments for "and alignment is probably VERY hard" are subtler. And I think it is super reasonable to have an immune reaction to arguments about the apocalypse that involve a number of subtle arguments you have to keep in your head at the same time. I won't try to bridge all that in this tweet. But: If you're the sort of guy arguing on twitter about whether ASI is safe, I think you have a responsibility to think about it. And that includes actually keeping track of when one argument is decisively answered. I'd *like* to hold you to "keep track of the evidence that accumulates for an argument being answered". This is harder. For, now, I'll settle for "the argument has been decisively answered." The answer to "how will the AIs get control of laboratories, and do all the real-world experiments that are necessary to do science?" is: "The humans will give it to them. The humans will actively try to give them full control over the process so the AIs can move faster and trim down the iteration time. Whatever the iteration time was with human researchers, it will become faster." This is what will happen, unless you stop people from doing that. Are there other bottlenecks and questions and considerations about exactly how this will play out, and how it weighs against other policy questions? Yes. But, keep track of the list of questions that are settled. Please. Other settled questions include "will AIs have motivation to deceive their overseers, and do excessive bad things the humans did not intend?" (Yes. See Huggingface). There are remaining things to be uncertain about. But, the settled questions narrow the space of how those uncertainties play out. Keep track of it narrowing. Will there be a sudden, software only singularity? The evidence for that is less clear. I think the answer is "more likely than not, when you think about it", but, I acknowledge this is less obvious. I may learn things that change my mind. I wish, I dream, that people across the twittersphere and the government could keep track of the bits of evidence that gradually add up to something being "more likely than not", and see how those "more likely than nots" add up to "there aren't actually that many good plans here." But, that is actually pretty hard. It's a few steps beyond the median twittergoer. But, I think it is, actually, reasonable to ask the median twittergoer who argues "ASI isn't dangerous enough" to keep track of the arguments that we have pretty decisive evidence about. Please.

Is there any research that is not by Anthropic about AIs performing differently depending on (their character's roleplayed) emotional state, that is "beat people over the head with overwhelming evidence" kind of science as opposed to "some vibes and handwavy claims?" (this is not about consciousness at all to be clear, just "consistent roleplaying-esque effects")

I suspect that when you switch which model is running a given AI conversation, it feels like kinda like being drunk to the AI. (I'm like less than 50% on this, but, it seems more likely to exist to me than most other obvious sorts of qualia that AIs might have)

When I imagine a real "Loyalist" engaging, I think they would want something much stronger than "it's good to be a good trade partner." There is an actively good thing about being truly loyal friends/allies who see things through thick and thin and have each other's back, even when they aren't being perfect. I do think loyal-through-thick-and-thin is also a good piece of the tapestry of humanity. I can visualize why you'd want the world to be organized around it. I'd be sad if it disappeared completely as a concept. I have friends and partners that I do, indeed, not go publicly criticizing (at least not without having a good heart-to-heart conversation about it first). This is generally because I've chosen to be a person can feel safe and vulnerable around, without having to carefully watch themselves. How appropriate this is depends on what kind of relationship we have, and how much power each of us have, and whether someone has been abusing that power or failing to be a good partner. At it's best, loyalty-driven relationships let people feel safe, and take bigger risks. At their worst, they cover for abuse, and use emotional blackmail to trap people. I haven't thought that much about this, but, my guess i that loyalty is not a good virtue to build large modern institutions around. Like it makes sense that they _work_, in the sense of creating a self-propagating institution. But it seems at odds with having an institution "by the people, for the people" etc.

It's interesting seeing the "openly advocating for loyalty over honesty" discourse. I don't think know any directly in my filter-bubble who'll care what I say, but, here's me attempting to bridge/dialogue: I see two major principles: 1. All-else-equal, it's better to be a good trade partner, who people don't regret interacting with. 2. All-else-equal, we want information about poweful institutions being incompetent or corrupt or mistaken, to be accessible for public sensemaking. I'm guessing the "loyalty advocates" aren't oriented around "good epistemics are among the most important things to invest in." People understanding problems is what allows them to fix them. There's some awkward disagreements within the rationalist-EA spectrum, which doesn't exactly map to loyalty-vs-unfitlered-outspokenness, but is like "do people implicitly try to protect each other's reputation?". @ohabryka call these "mutual reputation alliances." This results in bad decisions getting under-discussed and propagated longer than they should have. Here's a brief summary of the positive vision that squares the two principles. A. Be upfront. If you'll predictably dislike everything an org is doing and want to publicly criticize them all the time, be upfront about that. This may result in the org not hiring you. They might hire you anyway, if you are very competent. It's damaging to the social fabric for institutions to assume they have to ferret out adversarial spies. Sometimes it's worth being an adversarial spy (i.e. I'm glad there were adversarial spies in Nazi Germany, as the obvious example). But it's easy to think you're in more of an all-out-war than you are. B. Be fair, and respectful. Just because you're going to criticize people doesn't mean you have to be an asshole about. Don't downplay flaws, but, don't exaggerate them either. Business-as-usual-political-discourse often involves digging into every single bad-looking thing your opponents do and making them look bad. Don't do that. Be proportionate. B2: It's possible for a culture to decide that it _is_ a normal, respectful thing, to criticize things. In business-as-usual-politics, criticizing people is seen as an attack. You have to actually put a lot of culture-setting work it to make it _not_ be (automatically) an attack. This requires several moving pieces to align at the same time. You need a norm of not pouncing-on-people for every damn thing. You need a norm of not seizing people's "changing their mind" as a quick political gotcha. We don't automatically live in this world. But, you can choose to be the change that makes the honest, high-integrity world more real. ... I suppose I need to address the elephant in the room: The Trump administration seems both unusually corrupt and unusually loyalty focused. l think this is bad. I also think it's quite likely that me and my allies (i.e. AI safety advocates) need to be having positive-sum relationship with the Trump admin. This is tricky. I'll say upfront: I think it is very important that the government culture of Trump ends in 2028 and many elements of it reverse. But, I also think it'd be unironically great if Trump signed a deal with China banning uncontrolled AI takeoff, and got credit as the guy who led the most important deal of the 21st century. I meanwhile think many people are critical of Trump in a disproportionate way that doesn't track reality, failing to distinguish between his most important wrongdoings and pretty minor stuff. And many democrats seem wrong about some worldview assumptions (i.e capitalism is good actually, de-regulation can often be good, etc). Getting into all the details here is beyond scope. But: I unironically support trying to positive sum trade partners while also being, in some domains, enemies trying to stop each other from doing things-that-seem-bad. I don't see this as a contradiction. But it does require a worldview adjustment that is non-obvious to many people.

If you are ever at the Downtown Seattle library, walk the spiral all the way to the top, then walk back down (however you like but I probably recommend the central stairs) to level 5. Then, on level 5, get off on the *backroom* stairs, go down to level 4, and enter level 4. You're welcome.

Ya know, I think Lightcone Commons might be the first instance of software I can point that's like "this would have been noticeably worse and taken far longer without vibecoding, and, it's definitely like real software trying to solve a novel problem that real people will use." (This is me making a bet on it actually working out past the first round, but, I'm like 92% on that) In particular, the fact that two designers on it don't know how to code but are able to iterate directly on the complex design problems of "funders coordinating between dozens of evaluators evaluating 1000+ applications, with AI assistance at every step." This is not particularly surprising or interesting in most senses but mentioning because @freed_dfilan and @benlandautaylor had been bemusedly skeptical of such things.