Product #73

super:Link

https://open.substack.com/pub/productandrew/p/product-73?r=12u3a4&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Talking to Humans · Giff Constable · 2014

This book advocated for continuous customer discovery interviews long before Terresa Torres made the idea mainstream. It provides a practical and broad overview of how to do customer research. From identifying assumptions and shaping questions, to recruiting subjects and getting the most out of sessions. This is a great introduction to a subject that all product teams should be comfortable with.

Key Takeaways

Every new idea is built upon a stack of assumptions. Challenge your risky assumptions as soon as you can by getting out of the building and talking to users.

Be a detective. Look for clues that help confirm or deny your assumptions and allow them to influence what you do next. Customer discovery is about gaining deep insight into your customer, your partners, and your market.
Do customer discovery continuously. Don’t stop talking directly to customers. Your questions will evolve, but no matter what stage you are in, you’ll usually find that your best insights will come from talking to real people and observing real behaviour
To learn from your customer - you need to define them first. Creating a product for “everyone” is not actionable or useful. Get more specific. Who has the problem you’re trying to solve?
Business Assumption Exercise:

My target customer will be? How would you describe them?
The problem my customer wants to solve is? What do they struggle to do?
My customer’s need can be solved with? ‘Your elevator pitch’
Why can’t my customer solve this today? What obstacles prevent solving this already?
The measurable outcome my customer wants to achieve is?
My primary customer acquisition tactic will be? One channel will dominate.
My earliest adopter will be?
I will make money/ revenue by? One will dominate.
My primary competition will be? Think about both direct and indirect competition
I will beat my competitors primarily because of? How do you differentiate?
My biggest risk to financial viability is?
My biggest technical or engineering risk is?
What assumptions do we have that, if proven wrong, would cause this business to fail
Highlight assumptions that are highly important and uncertain. Not every assumption can be tested effectively through qualitative research.

Get Stories, Not Speculation. Humans are bad at predicting future behaviour.

Therefore “Would you buy this product?” is a terrible question.
Instead → Ask your interview subject to share a story about the past, that way you’ll be anchoring on real behaviour.

Ask Open-Ended Questions. Don’t talk too much. Get the subject to share openly. DO ask questions that start with ‘who, what, why and how’. DON”T ask questions that start with ‘is, are, would and do you’. Close with ‘What should have I asked you that I didn’t?’
Willingness to buy is hard to answer with qualitative research.

Instead try questions like...

How much do you currently spend to address this problem?
What budget do you have allocated to this, and who controls it?
How much would you pay to make this problem go away? (Don’t take answers too literally)

OR Set up a situation where the subject thinks they are actually buying something.

Get the most from feedback on Prototypes by separating prototypes from questions about their behaviour. Ask your questions about behaviour and challenges first. You don’t want discussion about product features to poison the conversation.
Design “Pass/Fail” Tests. Set goals for key questions and track results. Do so ahead of time. Targets can be an educated guesses. Setting a goal forces you to track what is happening. It helps you carefully think through what you are hoping to see, and will make decisions and judgment calls easier when reviewing data.
Once you know what you want to learn, think about how you might gather insight through observation (not just direct interviews)
If making a cold approach keep it concise, convenient, drop a name, follow up and practice your if you might have to leave a voicemail
Get Creative. Where are you customers? When are they likely to have time to talk? How can you convince them to talk?
Find the Moment of Pain. Connecting with people at the moment of their theoretical pain, can be very illuminating. Thinking about the moment of pain you want to address will lead you to find your consumers and you’ll be able to gather valuable observational research.
Make Referrals Happen. Use referrals to your advantage. If you want to talk to a doctor, use one to refer others. Set a goal of walking out of every interview with 2 or 3 new candidates.
How to ensure an effective session

Do interviews in person if possible
Talk to one person at a time
Add a note taker
Warm up and keep it human
Get them to tell a story
Don’t talk, listen.
Find out if they’ve hacked a solution
Parrot back to confirm understanding

The Truth Curve. You don’t know the truth about your product until people are using and paying for it. BUT don’t jump straight to a live product, that’s a very expensive and slow way to iterate on your idea. Start talking to customers and testing assumptions right away. As you build confidence increase the fidelity of your tests.

It is the customer’s job to explain their behaviour, goals, and challenges. It is the product designer’s job to come up with the best solution.

Full Book Summary · Amazon

Subscribe Button

In the News

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Article · Article

Quick Links

Cutler on Trees, Journeys and Loops · Article

How to Improve Retention the Ultimate Guide · Paywall Article

Customer Effort Score Defined · Article

Do you know the difference between ReCurring vs ReOcurring Revenue? · Article

How to communicate tradeoffs · Article

10 Usability Heuristics for UI Design by Jakob Nielsen · Article

How to create scalable systems · Article

How to succeed in MR.Beast Production · Paper

The Environment and Disease: Association or Causation? · Austin Bradford Hill · 1963

More often that not we are dependent upon our observation and enumeration of defined events for which we seek antecedents. In other words we see that the event B is associated with the environmental features A…. in what circumstances can we pass from this observed association to a verdict of causation? Upon what basis should we proceed to do so?

The Bradford Hill criteria help researchers determine if there is a causal relationship between an exposure and an outcome. There’s a lesson for Product Managers in here who are too quick to jump to conclusions…

Strength (or effect size): A small association does not mean that there is not a causal effect, though the larger the association, the more likely that it is causal.
Consistency (or reproducibility): Consistent findings observed by different persons in different places with different samples strengthens the likelihood of an effect.
Specificity: Causation is likely if there is a very specific population at a specific site and disease with no other likely explanation. The more specific an association between a factor and an effect is, the bigger the probability of a causal relationship.
Temporality: The effect has to occur after the cause (and if there is an expected delay between the cause and expected effect, then the effect must occur after that delay).
Biological gradient (dose-response relationship). Greater exposure should generally lead to greater incidence of the effect. However, in some cases, the mere presence of the factor can trigger the effect. In other cases, an inverse proportion is observed: greater exposure leads to lower incidence.
Plausibility: A plausible mechanism between cause and effect is helpful (but Hill noted that knowledge of the mechanism is limited by current knowledge).
Coherence: Coherence between epidemiological and laboratory findings increases the likelihood of an effect. However, Hill noted that "lack of such [laboratory] evidence cannot nullify the epidemiological effect on associations".
Experiment: "Occasionally it is possible to appeal to experimental evidence".
Analogy: The use of analogies or similarities between the observed association and any other associations.

Some authors also consider Reversibility: If the cause is deleted then the effect should disappear as well.

View the Paper

Book Highlights

The best moderators engage the user with the product with as little intervention as possible. They refrain from any commentary, and mainly observe and ask questions. Dan Olsen · The Lean Product Playbook

What is the smallest version of this I can produce to get useful feedback from others? Tiago Forte · Building a Second Brain

The scale obviously depends on the product and metric you choose. In some cases an improvement of 1% will be considered high impact, while in others 20% may be considered only a medium improvement. Itamar Gilad · Evidence-Guided

Even if the feedback can’t be used to infer natural labels directly, it can be used to detect changes in your ML model’s performance. Chip Huyen · Designing Machine Learning Systems

In two centuries, the human labor to produce a kilogram of American wheat was reduced from 10 minutes to less than two seconds. Vaclav Smil · How the World Really Works