imo there are broadly two views on the feasibility of AI safety for-profit models 1. You cannot internalize the externalities, so no incentives to pay for safety. In that world, hard to make the business work. 2. Safety is a core business blocker and you cannot externalize all the costs. In that world, safety is a multi-billion dollar market and safety companies like @GoodfireAI @Irregular or @ApolloResearch (biased obviously) are heavily underpriced. With all the cyber stuff, reward-seeking, reward-hacking, etc. recently, it seems to me like safety is now a core blocker for labs and willingness to pay for solutions should be in the billions. I think there are a lot of practical considerations, e.g. how does the external company sell "safety as a unit", but in principle the economics are there, no?

Only took me 8 years to hire Joschka! Among other things, our discussions about AI safety led him to switch to CS and ML back in the day :) Then during MATS he continued to do great work! Welcome to the team