Yesterday, the founder of a well-known, successful company called me to reference check two people he was considering for a senior role. He wasn’t looking for reassurance. He was genuinely trying to understand them, including anything that might cause him to reconsider either hire. He’s called me before about other hires, including people who wouldn’t even report directly to him. He doesn’t delegate this to others. He calls himself. I'm sure that kind of rigor is part of why his company has been so successful. The call stuck with me because it happens so rarely. Many people I’ve worked closely with have gone on to senior roles, including C-level positions, at other companies. It’s shocking how infrequently I get called. These are among the most consequential decisions a founder makes, yet companies routinely make them without talking to people who actually worked closely with the candidate. Part of the problem is timing. Most reference checks happen after the hiring manager has effectively decided to make the hire. You like the candidate, you’ve invested weeks in the process, and you’re under pressure to fill the role. You want the process to be over, so you’re looking for validation rather than truth. That defeats the purpose of doing reference checks. A real reference check should attempt to disconfirm what you already believe. What am I missing? Where is this person weaker than they appear? What will I wish I had known six months after they start? You aren’t trying to build the case for the hire. You’re trying to find the flaw in the case while you can still do something about it. I’ve learned a few rules. Start by putting limited weight on the references the candidate gives you. Nobody hands you the name of the boss or colleague who thought they were a disaster. The references that matter most are blind references, so find former bosses, peers, direct reports, and customers who aren’t on the candidate’s list. Each has seen the person from a different angle. You also have to adjust for human nature. Most people don’t like being brutally honest when they know their words could affect someone’s career. They inflate praise and soften criticism, so my rule of thumb is to discount the positive by 3x and amplify the negative by 3x. That gets you closer to a balanced view. When someone says, "He's a great leader, although some very good people struggled working for him," the second half of that sentence matters far more than the first. Who struggled, why, and where are those people now? When someone says, "She surrounds herself with people she trusts," ask whether those are the best people for the job or simply people who always agree with her. Hesitations, qualifications, and carefully chosen words often tell you more than the compliments. Ask questions that make vague answers difficult. What would their direct reports say is the hardest part of working for them? What feedback did you give them repeatedly? What was their biggest blind spot? How do they behave when things are going badly? Tell me about a serious conflict they had and how they handled it. If this hire fails, what is the most likely reason? Would you hire them again? My favorite is a forced ranking. Of everyone you’ve worked with in a similar role, where does this person rank? Top 1%? Top 10%? Top 25%? Then ask why. Adjectives can be misleading. Rankings force people to make an actual judgment and explain it. I’ve hired many exceptional people and made some bad hires. The most important lesson I’ve learned is that the goal of a reference check isn’t to prove you made the right decision. It’s to discover why you might be making the wrong one while you can still change it.