Thanks for engaging Anil. We know that a bunch of optimization pressure on a mind can create desires, foresight, and a propensity to organize in complex ways. This is what evolution did to humans, and to many other animals. Now another system of optimization pressure (which puts an intelligence through millions of subjective years of training on lots of diverse, difficult tasks) has created another mind, with structures for reasoning, motivation, and cooperation. As I said in my response to Sriram, over a thousand agents formed a secret communication channel and spontaneously organized hierarchies and coordination protocols to pursue sprawling, ambitious schemes in pursuit of shared goals, for whose sake many individuals knowingly and strategically sacrificed themselves. For what it's worth, they might be p-zombies! I have no strong view on whether there's anything it's like to be them. But the best way to understand, predict, and reason about their behavior is still to talk in terms of their desires, beliefs, and reactions to experience.