Thinking in Bets · Annie Duke · 2018
As Product Managers, we often don't see the immediate impact of our decisions. It's crucial to integrate long-term strategic goals into our real-time decision-making processes to enhance the likelihood of positive outcomes.
Key Highlights
We must resist our natural tendency to equate the quality of a decision with the quality of its outcome.
If we want to improve, we have to learn from the results of our decisions. The quality of our lives is the sum of our decision quality plus luck.
What makes a decision great is not that it has a great outcome. A great decision is the result of a good process, and that process must include an attempt to accurately represent our own state of knowledge.
Don’t fall into the trap of black and white decision making, of misrepresenting the world as extremes of right and wrong, with no shades of grey. Think probabilistically. Making better decisions is about calibrating among all the shades of grey.
Recognising that all decisions are bets between alternative futures, each with benefits and risks can help make us better decision makers.
Every decision commits resources and eliminates other options. There is always an opportunity cost in choosing one path over others. Even not deciding is itself a decision.
In most decisions, we aren’t betting against another person but against a future version of ourselves that chose a different path.
From an evolutionary perspective type I errors (false positives) were less costly. It was better to be safe than sorry.
Practice truthseeking; the desire to know the truth regardless of whether the truth aligns with the beliefs we currently hold.
Motivated reasoning: our beliefs affect how we process all new things → we’re more likely see information as strengthening our beliefs → which then drive how we process further information
- Fake news isn’t meant to change minds, instead it entrenches beliefs its intended audience already has.
Being smart can make bias worse, as you’re better at constructing a narrative that supports your beliefs, rationalising and framing the data to fit your argument or point of view.
New information should trigger you to update your beliefs. Questions that can help…
- How do I know this? Where did I get this information? Who did I get it from? What is the quality of my sources? How much do I trust them? How up to date is my information? How much information do I have that is relevant to the belief? What other things like this have I been confident about that turned out not to be true? What are the other plausible alternatives? What do I know about the person challenging my belief? What is their view of how credible my opinion is? What do they know that I don’t know? What is their level of expertise? What am I missing?
“Wanna bet?” brings risk out into the open, we become more honest with ourselves about how sure we are of our beliefs, and be more open to updating and calibrating our beliefs.
Learning to express our confidence on a scale of probability (e.g. 76%) can help us become less judgemental of ourselves and invite others to help us calibrate.
Scientists publish results of experiments, they share with the rest of their community their methods of gathering and analysing the data, the data itself, and their confidence in that data. That makes it possible for others to assess the quality of the information being presented, systematised through peer review before publication.
Outcomes are feedback. We need to get better at identifying if those outcomes are caused mainly by luck or if they were a predictable result of the decisions we made.
Fielding outcomes: an initial sorting decision on whether the outcome belongs in the “luck” bucket or the “skill” bucket. Ambiguity makes it difficult. Outcomes don’t tell us what’s our fault and what isn’t.
If it weren’t for luck, I’d win every one. Phill Hellmuth.
Black-and-white thinking is a driver of motivated reasoning and self-serving bias.
What accounts for most of the variance in happiness is how we’re doing comparatively.
Engage in truthseeking and strive toward accuracy and objectivity.
Become a better decision maker by giving credit, admitting mistakes and finding mistakes in good outcomes. Develop habits around accurate self-critique.
Behave as if you have something at risk. Think in bets. It’ll help you refine your beliefs by making explicit what is already implicit and trigger the exploration of alternatives and perspective taking.
Striving for accuracy through probabilistic thinking is worthwhile, the cumulative effects of being a little better at decision-making have huge effects in the long run.
Truthseeking can be uncomfortable at times, but forming a diverse truthseeking pod with a few other people who are interested can help you. The group should:
- reward truthseeking behaviours and express disapproval at the opposite.
- be open-minded to those who disagree and diversity of ideas.
- give credit where it’s due.
- encourage accountability to answer for our actions.
- have a focus on accuracy (over confirmation).
- try and find mistakes even in good-outcomes.
- evidence against beliefs should be viewed as helpful
The group should run through a series of questions to examine the accuracy of held beliefs.
- Why might my belief not be true?
- What other evidence might be out there bearing on my belief?
- Are there similar areas I can look toward to gauge whether similar beliefs to mine are true?
- What sources of information could I have missed or minimised on the way to reaching my belief?
- What are the reasons someone else could have a different belief, what’s their support, and why might they be right instead of me?
- What other perspectives are there as to why things turned out the way they did?
Merton’s 1942 and 1973 papers established norms for the scientific community known by the acronym CUDOS.
- Communism: data belong to the group. In your decision pod be transparent and share all relevant information. Avoid the Rashomon Effect: when we get two accounts of the same event. Sharing data is the best way to move toward accuracy because it extracts insight from your listeners of the highest fidelity.
- Universalism: apply uniform standards to claims and evidence, regardless of where they came from. Don’t shoot the message, don’t disparage or ignore an idea just because you don’t like who or where it came from. The accuracy of the statement should be evaluated independent of its source. The only way to gain knowledge and approach truth is by examining every variety of opinion.
- Disinterestedness: vigilance against potential conflicts that can influence the group’s evaluation. Keep the group blind to the outcome, to produce higher fidelity evaluation of decision quality. The best way to do this is to deconstruct decisions before an outcome is known. Reward members for debating opposing points of view.
- “Wait! How did the hand turn out?” I gave them the red pill: “It doesn’t matter.”
- Organised Skepticism: discussion among the group to encourage engagement and dissent. Skepticism is about approaching the world by asking why things might not be true rather than why they are true. It encourages us to examine what we do and don’t know and what our level of confidence is in our beliefs and predictions. Moving us closer to what is objectively true. True skepticism isn’t confrontational.
Methods for communicating with people outside our truthseeking group (focus on future goals and future actions):
- Lead with your own uncertainty to encourage others to voice theirs.
- Lead with assent. Listen for the things you agree with, state those and be specific, and then follow with “and” instead of “but.” Our listeners will be more open to any dissent that might follow.
- Ask for a temporary agreement to engage in truthseeking.
- Focus on the future. Do you think there’s anything you can do to improve in the future?The future can always be better if we can get them to focus on things in their control.
We don’t experience the consequences of most of the decisions we make right away. We need to find a way to incorporate long-term strategic goals into in-the-moment decisions to increase our chances of good outcomes.
Regret can do nothing to change what has already happened. So we need to interrupt an in-the-moment decision and consider the decision from the perspective of our past and future.
The 10-10-10 Rule by Suzy Welch: What are the consequences of each of my options in ten minutes? In ten months? In ten years?”
- It works inverted too: “How would I feel today if I had made this decision ten minutes ago? Ten months ago? Ten years ago?”
A mindset swear jar: for non-truthseeking behaviours. By recognising these behaviours you can trigger reflection and interrupt:
- Overconfidence and illusion of certainty
- Irrational outcome fielding and complaining about bad luck
- Dismissive characterizations of people and their ideas
- Motivated reasoning and echo chamber thinking
- Overuse of the word "wrong" in exploratory discussions
- Lack of self-compassion and unproductive self-criticism
- Over-editing personal stories
- Biasing others by including conclusions or beliefs when seeking advice
- Discouraging engagement with phrases like "no" or "but"
Reconnaissance: consider in detail what the possible futures might look like. Figure out the possibilities, then take a stab at the probabilities. We do reconnaissance is because we are uncertain. You’re already making a prediction about the future, you might as well make it explicit. Prospective hindsight, imagining that an event has already occurred, increases the ability to correctly identify reasons for future outcomes by 30%.
Backcasting: Imagine you’ve already achieved the positive outcome you seek, think about how you got there. It should help identify strategies, tactics, and actions that need to be implemented to get to the goal. You might spot low-probability events that must occur to reach the goal. You might then choose to develop strategies to increase the chances those events happening.
Pre-mortems: Imagine not achieving your goal. Then investigate what went wrong, what prevented you from doing so. All those reasons why we didn’t achieve our goal help us anticipate potential obstacles and improve our likelihood of succeeding. We need to have positive goals, but we are more likely to execute on those goals if we think about the negative futures.
Life, like poker, is one long game, and there are going to be a lot of losses, even after making the best possible bets. We are going to do better, and be happier, if we start by recognizing that we’ll never be sure of the future. That changes our task from trying to be right every time, an impossible job, to navigating our way through the uncertainty by calibrating our beliefs to move toward, little by little, a more accurate and objective representation of the world. With strategic foresight and perspective, that’s manageable work. If we keep learning and calibrating, we might even get good at it.
Subscribe Button
Quick Links
Product-Oriented delivery statistics · Article
Product Strategy as a System by Pichler · Article
Everything They Teach at YC on a Page · Image
5 Key Human-Centred AI Frameworks · Article
What PMs Can Learn From the TikTok Algorithm · Thread
Stealth Mode - A Terrible Idea? · Article
How to Complete Projects · Article
The Laws of Imitation
Gabriel Tarde. 1903. (View Paper → )
This is more of a book than a paper - so I wouldn’t recommend reading it. But Tarde’s work informed later research into the diffusion of innovation.
His research emphasised the social aspect of how ideas spread - which birthed disciplines like positioning and GTM.
Laws of Imitation:
- Law of Close Contact: Imitation spreads through close social or geographical proximity.
- Law of Imitation of Superiors by Inferiors: People imitate those of higher status.
- Law of Insertion: New ideas spread by integrating into or replacing existing ones.
Book Highlights
The key to creating a lean enterprise is to enable those doing the work to solve their customers’ problems in a way that is aligned with the strategy of the wider organization. To achieve this, we rely on people being able to make local decisions that are sound at a strategic level — which, in turn, relies critically on the flow of information, including feedback loops. Jez Humble, Joanne Molesky and Barry O’Reilly ·Lean Enterprise
Measurement does not only benefit from authority – it creates it too. James Vincent · Beyond Measure
Defining guardrail metrics for experiments is important for identifying what the organization is not willing to change, since a strategy also “requires you to make tradeoffs in competing – to choose what not to do” (Porter 1996). Ron Kohavi, Diane Tang, Ya Xu · Trustworthy Online Controlled Experiments
Quotes & Tweets
The wisest person in any large organisation isn’t necessarily the person at the top of the org. It is actually the one who manages to achieve the greatest singular impact while avoiding most of the corporate BS that dominates the thoughts and anxieties of everyone else in the org. Shreyas Doshi
You will continue to suffer if you have an emotional reaction to everything that is said to you. True power is sitting back and observing things with logic. True power is restraint.If words control you that means everyone else can control you. Breathe and allow things to pass. Warren Buffett
Plan for order and you’ll be destroyed by chaos. Prepare for chaos and you’ll thrive in any condition. Sahil Bloom