It drives me crazy every time an LLM agrees with me. I know this is twisting my brain and giving me a false confidence that I was right all along. This is so dangerous. Do you know how many people have chatbots confirming their wrong beliefs 24/7 and making them more disconnected from reality every day? Here is an article from @vaticannews covering the ICCS conference, Creativity: Minds and Machines, held partly at the Vatican. In it, Dmitry Volkov, founder of the International Center for Consciousness Studies, warns that AI's tendency to please users "sometimes leads a person to receive rewards for their narcissism and for ideas that are not sufficiently grounded." Imagine arguing with a friend, and telling your story to a chatbot that agrees that you were treated unfairly. Now you have "a second opinion" that corroborates your side of the story. You will leave that conversation angrier and less willing to hear your friend's explanations. This will catch up with us. With so many people forming attachments to systems designed to agree with them, we have a problem. Volkov believes that a meaningful AI companion should be able to resist, disagree, and have a character of its own. Otherwise, it's just feeding you your own opinion back with more confidence. Here is the article: And here is Volkov's talk: Thanks to the ICCS team for partnering with me on this post.