Superforecasters are individuals who consistently make remarkably accurate predictions about future events, outperforming experts and intelligence analysts through learnable techniques rather than innate genius. They excel by adopting a fox mindset (pragmatic, versatile, and open to changing their minds) rather than a hedgehog approach that views everything through one rigid lens. Most importantly, they maintain a growth mindset and commitment to self-improvement, treating failures as learning opportunities.
The core method involves breaking down complex questions into manageable components, distinguishing the known from unknown while scrutinising all assumptions. Start with the outside view by finding base rates and comparison classes; nothing is 100% unique. Then apply the inside view to identify what makes this situation special. Synthesise these perspectives along with diverse viewpoints from others, leveraging the wisdom of crowds. Express predictions using precise probabilities, debating differences as small as 3% versus 4%, as granularity predicts accuracy.
Successful forecasters update frequently but incrementally, making small adjustments as new information arrives. This follows Bayesian belief updating, adjusting confidence proportionally to the weight of evidence rather than overreacting or underreacting. They view problems from multiple angles simultaneously like a dragonfly, actively seeking point-counterpoint discussions and acknowledging counterarguments. When uncertainty is high, they stay cautious within the 35-65% range, moving out tentatively.
Tracking predictions rigorously is essential for both assessment and learning. Use specific terms, timelines, and numerical probabilities to enable Brier scoring, which measures both calibration and resolution. Conduct unflinching postmortems on successes and failures alike, asking where exactly you went wrong. Regular feedback, like that received by meteorologists and bridge players, prevents overconfidence and accelerates improvement.
In teams, combat groupthink by fostering constructive disagreement and ensuring people form judgments independently before aggregating. Use precision questioning to dissect vague claims and pre-mortems to surface doubts safely. Apply mission command principles; communicate goals clearly but let those closest to the action determine methods, as no plan survives contact with reality unchanged.
The Ten Commandments codify these practices:
- Triage, focus on questions where effort pays off, avoiding both trivial and impossible problems.
- Decompose intractable problems using Fermi-style analysis.
- Balance outside and inside views appropriately.
- Update thoughtfully to new evidence.
- Identify clashing causal forces.
- Distinguish degrees of doubt granularly.
- Balance confidence and prudence.
- Learn from errors without hindsight bias.
- Leverage collective wisdom through perspective-taking and constructive confrontation.
- Master the error-balancing bicycle (continuously adjusting between leaning too far left or right), each commandment requires navigating between opposing mistakes.
- Don't treat commandments as rigid rules, maintain constant mindfulness and adapt guidelines to context.
While predictability has limits, especially with black swan events and nonlinear systems, we shouldn't dismiss all prediction as futile. Superforecasting isn't about prophecy but about hard work: careful research, self-criticism, granular judgements, and relentless updating. Anyone can develop these skills through deliberate practice with clear feedback, gradually getting closer to the truth by thinking in small units of doubt.
Quick Links
Great example of craft from the early days of Gmail · Video
Curiosity vs Criticality: From ‘How might we?’ to ‘At what cost?’ · Article
Operating on the edge of competence · Article
A Periodic Table of System Design Principles · Article
What do Prototypes Prototype?
Stephanie Houde and Charles Hill. 1997. (View Paper → )
We propose a change in the language used to talk about prototypes, to focus more attention on fundamental questions about the interactive system being designed: What role will the artifact play in a user's life? How should it look and feel? How should it be implemented? The goal of this chapter is to establish a model that describes any prototype in terms of the artifact being designed, rather than the prototype's incidental attributes By focusing on the purpose of the prototype—that is, on what it prototypes—we can make better decisions about the kinds of prototypes to build. With a clear purpose for each prototype, we can better use prototypes to think and communicate about design.
This paper shifted the industry conversation away from describing prototypes by their medium (e.g., "paper prototype" or "high-fidelity prototype") toward their purpose: testing role (what the product does for users), look and feel (user experience), or implementation (how it works technically). This framework remains fundamental in UX and product development, influencing how teams plan and communicate about prototypes.
Key Takeaways for Product Managers:
- Focus on purpose over medium: Choose prototyping methods based on what questions need answering, not arbitrary notions of fidelity.
- Different audiences need different prototypes: Technical teams, executives, and users may each require different prototype types.
- Build multiple prototypes: Complex products often require separate prototypes for role, look and feel, and implementation rather than attempting one all-encompassing prototype too early.
- Be explicit about what's being tested: Clearly communicate what aspects a prototype addresses and what it doesn't to avoid misinterpretation.
The model continues to help PMs make more efficient prototyping decisions, saving time and resources while answering the most critical product questions at each development stage.
The model presents a triangular framework for understanding what aspects of a design a prototype is meant to explore.
The Three Dimensions
Role: Explores questions about what function the product serves in a user's life and how it's useful to them. Role prototypes answer questions like: What problem does this solve? How will it fit into users' workflows? What value does it provide?
Look and Feel: Explores the concrete sensory experience of using the product - what users see, hear, and feel when interacting with it. These prototypes answer questions about visual design, interaction patterns, and the overall user experience.
Implementation: Explores the technical underpinnings - how the product actually works "under the hood." Implementation prototypes answer questions about technical feasibility, performance, and architecture.
The centre of the triangle represents a fourth category:
Integration: Explores the balance between all three dimensions, representing a more complete experience. These prototypes attempt to address role, look and feel, and implementation simultaneously.
How to Use the Model For any given design problem, the model helps designers:
- Identify which dimension(s) contain the most important open questions
- Create prototypes specifically targeted at answering those questions
- Communicate clearly to stakeholders which aspects of the design are being explored (and which aren't)
The model suggests that early in the design process, it's often more efficient to create multiple targeted prototypes addressing different dimensions rather than attempting to create a single comprehensive prototype that tries to do everything at once. Each prototype is placed on the triangle based on which dimension(s) it primarily addresses, creating a visual way to communicate a prototype's purpose to team members and stakeholders.
/w=3840,quality=90,fit=scale-down)
Book Highlights
Checklists can motivate new users to complete crucial set-up tasks. Checklists can turn complex, multi-step processes—such as scheduling a month of social media content—into simple, achievable tasks. Onboarding checklists employ the Endowed Progress and Zeigarnik Effect. Wes Bush · Product-Led Growth
He argued that for all the danger the technology posed, its real message was one of hope, for “a new interpretation of man, of man’s knowledge of the universe, and of society.” M. Mitchell Waldrop · The Dream Machine
A reference class is a group of comparable situations or companies or challenges Richard Rumelt · The Crux
/w=3840,quality=90,fit=scale-down)
Quotes & Tweets
If you do not know how to ask the right question, you discover nothing. W. Edwards Deming
When we work in silos, we do more work, we take longer to complete that work, and we often build sub-optimal solutions Teresa Torres