Product #34

Author

Andrew Clark

Year

2023

super:Link

https://productandrew.substack.com/p/product-34

The Right It · Alberto Savoia · 2019

Marty Cagan stated that most product ideas won't meet your expectations, and even those that do will require multiple iterations to get there. These two principles are incredibly powerful in the realm of Product Management. If we accept them as true, the theory of product management starts to make sense. In his book 'The Right It,' Alberto Savoia brilliantly articulates these principles as the law of market failure. He offers a robust approach to navigate this perilous landscape and determine what to build. Just when you might feel doomed to fail, Savoia's insights provide a guiding light. Let's dive into his perspective and uncover the path to success.

Key Highlights

Make sure you’re building ‘The Right It’ before you ‘build it right’.
Most new products will fail in the market even if competently executed. Great execution can’t save you if you’re building ‘The Wrong It’.
‘The Right It’ is defined as an idea for a new product, that if competently executed, will succeed in the market.
The author believes in testing ideas in a the market quickly. Time spent theorising is labelled being in ‘ThoughtLand’. You can’t tell if something is ‘The Right It’ in ThoughtLand.
Other peoples data and opinions are not valuable, you need to collect your own data (YODA)
Your market data must come with skin in the game. Time, money or reputation from your prospects,
Thinking tools can help you articulate your idea as a testable hypothesis

Frame your idea as a Market Engagement Hypothesis in the XYZ format:

At least X% of Y will Z.
Where Y is a clear description of your target market and Z is how and to what extent the market will engage with your idea.
Then find a smaller more accessible market where you can test that same hypothesis

The author labels certain types of prototype ‘Pre-to-types’. These tools can quickly validate your market engagement hypothesis → they answer “Should we build it?”

PreToType examples: Mechanical Turk, the Pinocchio, the Fake Door, the Facade, the YouTube, the One-Night Stand, the Infiltrator, the Relabel

Weigh evidence based on how much skin-in-the-game people provide

E.g. ignore opinion but value pre-orders
There’s a hierarchy: Money > reputation > time > everything else

Key tactics and mindsets:

Think globally, test locally. Find an accessible local market that provides valuable information about the larger market you wish to dominate.
Think cheap, cheaper, cheapest. Look for the cheapest way to test your idea.
Testing now beats testing later.
Tweak it and flip it before you quit it. You need to iterate to know if it’s the idea or the execution that’s the problem.

The end to end process:

Start with an idea
Identify the Market Engagement Hypothesis
Turn the MEH into a say-it-with-numbers XYZ Hypothesis
Zoom in, and test your XYZ hypothesis in a local market
Use preToTyping techniques to run experiments and collect YODA (your own data)
Use Skin-in-the-Game to weigh YODA.
Decide on the next step (go for it, drop it or tweak it)

Follow these steps reduces your probability of failure. If you fail you’ll fail well. If you stick with it, you’ll find an idea that works in the end.

Full Summary · Buy it · More Summaries

Subscribe and share Product so you and your team can keep pace
with Product Management and related fields.
< Subscribe Now Button >

In the News / Quick Links

Paul Adams (Intercom CPO) talking AI product strategy · Podcast
Tech predictions for 2024 and beyond · Article
Vitalik joins the techno-optimists as the effective altruists retreat · Article
How AI could lead to a better understanding of the brain · Nature Article
How Google does code reviews · Article
Peer-to-peer networks use something called the gossip protocol · Article

The Science of Usability Testing · Jean E. Fox · 2015

As well as the insights below, this paper gives you a great primer for how usability testing has evolved over time.

Number of participants

There are diminishing returns to increasing participants. Around four to five participants can reveal about 80% of usability problems, with diminishing returns for additional participants.
You’ll need more participants if you’re testing a complex system, or if your user group is not heterogenous.
General guidelines suggest 5-10 participants per user group for qualitative tests and 20-30 for quantitative tests.

Number of Trained Observers

A single observer might miss issues that multiple evaluators could identify.
The number of trained observers (evaluators) significantly affects the identification of problems in usability testing. The lesson here is to involve the team.
Adding more evaluators can be as effective as adding more participants, especially when participant recruitment is challenging or time is limited.

Use of the Think-Aloud Method

The think-aloud method can influence test results; for example, it can make participants more aware of their thought processes, potentially leading to faster problem-solving.
Different approaches to think-aloud (e.g., traditional vs. relaxed methods) can have varying effects on aspects like task time and mental workload.
While think-aloud is a powerful tool for identifying usability problems, its implementation should be carefully considered, especially in tests focusing on quantitative measures.

View the Paper

Book Highlights

Insert Artwork

“Every time we do this we find holes. We find things that we thought another team should be taking care of, but it didn’t know. We find the necessary stuff in between the big important features that we’d forgot to talk about.”
Jeff Patton · User Story Mapping