What is it about? Was it worth reading?
Teressa Torress is a household name in the Product community, and this might be the most influential product book written for a decade. It became an instant classic with Product Managers for good reason.
I suspect it resonated with PMs because so many teams are struggling to do product discovery well. Teressa made it seem easy.
Once you’ve mapped your opportunity space as a tree, prioritisation and research become easier. You’ll be able to systematically identify the most important thing to do next (research, experiment or build).
Meeting customers regularly increases your both the quality and quantity of insight, and helps you navigate your opportunity space.
There’s a ton of value in this book, so I recommend reading it in full.
You Might Also Like…
Book Review and Summary: Talking to Humans by Giff Constable
Get a comprehensive review and summary of the best-selling book "Talking to Humans" by Giff Constable, the market research and product development expert. Learn about the key principles and practices for conducting effective customer research and using the insights to inform product development.
Book Review and Summary: Testing with Humans by Ron Chernow
Get a comprehensive review and summary of the best-selling book "Testing with Humans" by Ron Chernow, the market research and product development expert. Learn about the key principles and practices for conducting effective user testing and using the insights to inform product development.
Holistic Product Discovery
This is a great product discovery reference book. It brings together product discovery and delivery, and provides a great overview of the theory to date. It’s a book without ego, introducing just a couple of frameworks whilst signposting other important work. It doesn’t quite achieve a single theory that ties everything together, which is a shame. I do love though that it assumes a little product knowledge, which is refreshing as it allows for brevity. Well worthy of a read.
The 20% that gave me 80% of the value.
- Agile helped teams build more software - but it agile doesn’t help us build the right thing. The fruit of discovery work is the time saved not building something that won’t work.
- Historically, teams viewed discovery as as a one-time activity at the beginning of projects - and focused too much on delivery. Teressa advocates for continuous discovery → having weekly touch-points with customers to conduct small research activities - with a desired outcome in mind
- Successful products provide value to both the business and the customer. How we frame a problem influences how we solve it - therefore teams should frame problems in a customer centric way.
- Mapping the opportunity space and selecting which opportunities to pursue are high leverage points in a product team’s system, they are crucial to a team’s success.
- The Opportunity Solution tree helps teams select the most impactful thing to do next - even if that activity is more discovery/exploration. What should we build next? Becomes what should we do next?
- Prioritising opportunities not solutions will save you time. Product strategy doesn’t happen in the solution space, it happens in the opportunity space.
- Teams should be assigned outcomes, and given freedom to explore and select opportunities. It’s unfair to assign an outcome to a team they can’t influence. It’s important to help teams find tractable metrics, that will ultimately impact the business outcome.
- Visualise what you know in a customer’s experience map - and use it to guide your interviews. Interviews help discover and explore opportunities. Meet with customers regularly and you don’t have to cover so much ground. Instead you can get insight into the opportunity area that you’re working on now - just in time to influence what potential solutions could look like
- You and your team can take a step towards this way of working today. Pick your moment to advocate for more discovery. To maintain autonomy teams must show their work to stakeholders - opportunities explored, customers spoken to, metrics identified etc.
Longer form notes, typically condensed, reworded and de-duplicated.
Continuous Discovery Habits
- The book describes a continuous discovery framework that’s equally applicable to new products and iterating on existing ones.
- The Two Types of product work
- How do you know you’re making a product or service your customers want?
- How do you guarantee you are creating value for customers in a way that creates value for the business?
- Delivery: work you do to build and ship
Think about questions like:
- Project Management Approach: Business owners decided what to build. Discovery happened infrequently (with the yearly planning cycle.) Projects with fixed timelines were assigned to engineering teams. Project managers managed the work, budget and schedule
- Challenges: Software development is unpredictable. Often over time and over budget. Needs of the business trumped needs of the customer. Teams learn late that customers don’t like their product. Lots of waste.
- Agile and UX Approach: Projects were too big and teams were building the wrong stuff. Agile advocated for shorter cycles, sustainable pace, maximum flexibility, simplicity. Rise in the adoption of Scrum and Kanban. Growth of user-experience design and user research → to collect customer feedback
- Benefits: Shorter development cycles, Better product metrics. Better at usability. Better at iteration and starting small.
- Challenges:Leaders didn’t want to give up discovery. Stakeholders cling to their ideas still. Teams are bad at estimating. Typically the whole organisation didn’t become agile - stuck in old cycles. Teams weren’t allowed to fail - had to deliver on time and on budget even if they know their thing wasn’t valuable. Usability testing done late, user research outsourced, teams measured by what they delivered not value created.
- Still too much time was wasted. Teams learned after shipping code they’d built the wrong stuff. Instrumentation showed us that our new stuff didn’t work
- The future: Product teams adopting continuous discovery habits. Engaging with customers to understand their wants and needs. Co-creating products with them. Teams bridge the gap between what’s possible to build and what customers need.
Continuous Discovery Mindset (Required before you start)
Working definition of Continuous Discovery
Product teams adopting discovery habits in a structured, and sustainable way, enabling them to continuously infuse their product decisions with customer input. Fresh customer input helps with decision making.
A Common Framework for Continuous Discovery
- Balance the needs of the business with the needs of the customer. Profit shouldn’t come at the cost of serving the customer. Serving customers is how you generate profit.
- Begin with the end in mind. Shifting from output mindset to an outcome mindset. Lay the foundation for success - by starting with outcomes not outputs
- Task product trio with delivering an outcome. Give them space to explore the best outputs. Give the team the latitude thy need to create value. They have a choice - choose to engage with customers - do the work to understand the customer context.
- Organisational context has a big impact on success. Will the team choose to put the customer first? Or take shortcuts and do things the business wants without regard for the customer
The Challenge of driving outcomes
- The purpose of the customer touchpoint is to conduct research in pursuit of a desired outcome. Looking for a ways to serve customers - that create business value.
- Finding the best way to achieve that desired outcome is an ill-structured (or wicked) problem. There are many potential solutions, no right or wrong answers only better or worse ones.
- Much of the work needs to be about framing the problem. Frame the problem in a customer centric way! This avoids creating features that work only for the business.
- Opportunities are customer pain points and desires
- Opportunity Space = Problem Space = Desire Space
- To reach a desired outcome - a product trio must discover and explore the opportunity space. The opportunity space is infinite. Problem framing is therefore important - how we frame a problem influences how we solve it. Try out many framings - explore how each impacts the solution space
- Two of the most important steps for reaching a desired outcome are first
- How we map out and structure the opportunity space
- How we select which opportunities to pursue
The Underlying Structure of Discovery The Opportunity Solution Tree
The Opportunity Solution Tree is a visual representation of the paths you might take to reach a desired outcome.
Benefits of an Opportunity Solution Tree
Starting with the business need means everything is in service of the business. Filtering by customer opportunities ensures everything is in service of the customer.
Stops the team settling on the first or most obvious solution. Helps the team generate more options (How else might we solve this problem?). Opinion battles don’t encourage collaboration, the tree does. Builds a shared understanding of the options.
Helps break down a project (outcome or opportunity) into smaller chunks. Teams can focus on solving one thing at a time. Which helps deliver value every sprint
- Stops you bouncing from tactic to tactic, you can take a more strategic approach to selection
- looking at a problem too narrowly
- looking for evidence that confirms our beliefs
- short term emotions affect our decisions
- Inspires a compare and contrast mindset - not a whether or not mindset. Which of these customer needs is more important for us to address right now? What else could we build?How else might we address this opportunity?
- Good discovery only reduces the chance of failure - be prepared to be wrong - balance having confidence with doubting what you know.
- Visualises the decision making process
The problem space and solution space evolve together - as you explore solutions you learn more about the problem. If something doesn’t work...
- Based on my understanding of the customer I thought this would work. What did I misunderstand about my customer? How might we revise the opportunity space now
Helps us know if we’ve gone wide enough, if we are doing enough assumption testing, if we have enough solutions mapped.
Helps justify decisions to leadership. Show your work. Shares the right amount of detail with leadership.
How to Focus on Outcomes not Outputs
- Product teams will need to do discovery work to identify connections between product outcomes (metrics they want to influence) and business outcomes (metrics that drive the business)
- Translate business outcomes into product outcomes that you can deliver.
- Negotiate appropriate product outcomes with leadership
- Determine when to set learning goals versus performance goals
- Focusing on outcomes is hard
- Don’t know how to measure the outcome
- Don’t know how to impact it - make it tractable
- Or if it’s the right outcome to pursue the first place. Managing by outcomes is only effective if you choose the right one
- 90 day retention was a sensible yardstick. But it took too long to iterate against. The team kept shortening (90→ 30 → 5). They were able to experiment faster, but they weren’t sure if they were driving the business outcome.
- Customer interviewers uncovered 2 churn factors
- Not all customers understood the value of tailor-made dog food
- Some dogs didn’t like the food
- The team now had two things to action and measure - that they knew would drive business value
- They needed to increase the perceived value of tailor-made dog food
- They needed to increase the number of dogs that liked the food
- It’s hard to use lagging indicators to drive work - you can’t the impact of your work quickly
- Explore leading indicators that predict the direction of lagging indicators
- Assigning a leading indicator is better than assigning a lagging indicator
Measure business value
Measure how the product drives business value
% Dogs who like the food
Tracks usage of specific features
Owners who use the transition calendar
- Assign a team a business outcome - they might not be able to influence it without others ☹️
- Assign a team a product outcome - great - flexibility and autonomy to the team 🙂
- Might be OK for a junior team
- Might be OK for mature products with proven traction metrics (previously done discovery)
- Benefits of Focusing on Product Outcomes
- Autonomy - Gives the team autonomy to find the best solution (they have the expertise)
- Anti-Fragility - you don’t need to know the solution, or you can be wrong
- Measurement - becomes an important part of what you do
- Teams make faster progress - don’t have to wait for lagging metric to work through
- Control - teams are in control of Product Outcomes
- Reduces communication overheads
- Product leader brings the business view - what’s most important for the business
- Product trio bring the customer and technology knowledge
- Negotiation typically includes discussion on:
- Expected gains, risk, resources, approach, removing competing priorities, time to expected results, discovery time.
- If the business want more impact. That normally results in bigger bets and a bigger chance of failure.
- During the negotiation - we shouldn’t be talking about solutions - this should emerge from discovery
- Specific, measurable, achievable, relevant and time-bound goals
- Teams that set specific, challenging goals outperform those that don’t
- They create focus, inspire effort and persistence and help surface relevant organisational knowledge
- They have to believe they can achieve it
- They need to be committed to the goal
- Teams need to be involved in their own outcomes
- If the team doesn’t have the strategies for how to achieve them, they can be demotivating
- Internal learning goals are more effective than performance goals
- Getting teams to do their best, was more effective than setting specific goals
- Unless you can give them the strategies they need
- Start with a learning goal - discover opportunities and drive engagement
- Then task with a performance goal
- E.g → Learn what leads to churn → Start to tackle it
- OKRs are one way of managing based on outcomes
- Shift your product leader toward an outcome mindset. Situation ↓ Response.
- Ask for more business context. Explore it:
- Who is the target customer?
- What business outcomes are we trying to drive with this initiative?
- Why do we thing this initiative will drive that outcome?
- Check if the product outcome is really a leading indicator of the desired business outcome
- Map out the product outcomes that might drive the business outcome
- Give feedback to your leaders
- If given a product outcome - ask for the business context
- What business outcomes are we trying to drive with this product outcome?
- Clearly communicate how far you think you can get in the allotted time
- Try and shift towards a two-way negotiation
- Before setting your outcome - ask for more business context
- What’s the most important thing to business right now?
- Try to frame this conversation in terms of business outcomes
- Is there a customer segment that is more important than other customer segments?
- Are there strategic initiatives we should know about?
- Use the information you gain to map out the most important business outcomes and what product outcomes might drive those business outcomes
- Get feedback from your leader
- Choose a product outcome that your team has the most influence over
- Check you’re being tasked with a product outcome, not a business outcome or a traction metric
- If you are being tasked with a traction metric - is it proven?
- If this is the first time you’re working on a new metric, do you have a learning goal?
Discovering Opportunities - Opportunity Space
The Experience Map - Visualising What You Know
- Map out your own perspective first
- Explore everyone else’s perspective - together you’ll have a richer understanding
- Merge them into a shared experience map
- Make explicit what you know, what you don’t, what your hunches are
- Note where you have open questions for customer interviews
- The map will guide your customer interviews
- Help you structure the opportunity space
- Start with the desired outcome - focus on a question
- E.g: what’s preventing our customers from completing their application today?
- The outcome constrains what you capture
- Think about how broad or narrow to go
- If focused on optimisation - narrow is OK
- If focused on open-ended outcome - expand the scope
- E.g for Netflix: increase hours watched - set broad scope
- Not as broad as ‘entertainment in general’
- But as broad as through satellite, twitch, YouTube and competitors
- Constrain scope, but not too much
- Steps to building your experience map
- Start individually to avoid groupthink - Develop your own perspective
- Draw from your customers perspective
- Not everything - just what’s relevant
- Think about acquisition, retention, work of mouth etc.
- Language is vague - drawing makes your ideas more concrete
- Take turns to share drawings
- Ask questions - explore your team mates perspective
- Pay attention to the differences - be curious
- Share your point of view - don’t advocate
- Answer questions to clarify your thinking
- Synthesise your work - don’t pick the best
- Start by turning each of your individual maps into a collection of nodes and linked
- Node - distinct moment in time, action or event
- Links - what connect notes
- Links - might be a loop
- Create a new map that includes all of your individual notes
- Collapse similar notes together
- Determine the links between each note
- Add context (what are they thinking, feeling, doing)
- Avoid these common anti-patterns
- Getting bogged down in debate
- Using words instead of visuals
- Moving forward as if your map is true
- Forgetting to refine and evolve your map as you learn more
- What factors do you consider when buying jeans? → Fit
- Tell me about the last time you bought beans? → On amazon
- How did you know they would fit? → I didn’t, they were a brand I liked and they were on sale
- People struggle to answer direct factual questions accurately
- People give answers that align to their identity not behaviour
- We have a tendency to rationalise our behaviour - your brain summons a compelling reason and it feels like the truth
- Be wary of a coherent story, told by both the thought leaders in our space and by our customers themselves
- Speak to customers occasionally, you’ll have to ask them about everything. Speak to them weekly, you can ask about what you need to learn about in that moment. I call this concept just in time learning and it’s great - I do it with books
- Balance each interview…
- Broad exploration into needs, pain points and desires
- Diving deep on the specific opportunities that are most relevant to you
- Don’t explore anything that isn’t valuable to your customer!
- Primary Research Question: What needs, pain points and desires matter most to this customer?
- Memories about recent instances are more reliable than our generalisations about our own behaviour or our answers to direct questions.
- Tell me about the last time you did x. Asking about a specific instance better reflects actual behaviour, not perceived behaviour
- Think about how broad your question should be. Do you care about your product or also your competitors? Or tangential products and experiences
- Narrow is good for optimising your product. Broad is good for new opportunities.
- If you ask a short question, people will give short answers. They’ll match you.
- Use Temporal prompts - Start at the beginning, What happened first?
- Where were you? What happened next? What happened before that? Who was with you? What problems did you face? How did you overcome that challenge?
- Bring them back to this instance, instead of letting them generalise. In this specific example, did you face that challenge?
- Let the participant talk about what they care about most
- Don’t force it. The stakes are low. It’s OK if you don’t get what you need. Especially if you’re interviewing regularly
- Interviewer Snapshots turn detailed notes into insights. A one page synthesis of what you learnt in each interview
- Interview Snapshot
- Quick facts - Customer Segment etc
- Insights - weaker signal opportunities, turn into opportunities over time
- Opportunities - need, pain point or desire
- Memorable quotes
- Experience map will help guide each interview
- Is their story the same or different to your experience map?
- Draw the underlying structure of the stories that you hear - the nodes and the links
- Helps explore an ever-evolving opportunity space
- Needs change
- Markets change - competitors, products
- A digital product is never done - the opportunities space is never finite or complete
- Just because you chose a target opportunity to build, doesn’t mean that you can stop interviewing
- As it assumes you chose the right one - and you’ll be able to address it
- Interviewing is not a step in a linear process - instead our goal is to interview continuously
- Enables you to revisit the opportunity solution tree
- It’s mush easier to maintain a habit than to start and stop it
- The goal is to wake up with a weekly interview scheduled without you having to do anything
- The goal is to make it easier to interview, than to not, by the use of automation
Recruit participants while they are using your product or service
- Do you have 20 minutes to talk with us for £20?
- Or ads to drive traffic to a landing page
Ask your customer-facing colleagues to recruit
- Agree triggers
- Customer calls to cancel
- Customer has a question about feature x
- Customer requests a customisation schedule an interview
Make it clear who you want to talk to, make it easy to schedule
- I’d love for you to share your feedback with our product team, can we schedule 20 minutes for you to talk with them?
Interview your customer advisory board
- If hard to reach, or you have a small market, or their time is too valuable…
- Setup a customer advisory board
- Use them as participants
- Offer an ongoing incentive
- Scale it to suit the the number of interviews you need
- Get all the trio to do it - don’t have a single person be the voice of the customer
- You’ll each hear different things - different perspective
- Relying on one person to recruit interview participants
- Asking who, what, why, how and when questions. Use story based instead
- Interviewing only when you think you need it
- Sharing what you learned by sending out pages of notes or a recording - use snapshots instead
- Stopping to synthesise a set of interviews - just keep interviewing at a sustainable cadence
Mapping the Opportunity Space
- With each interview the structure of the opportunity space continues to evolve. Keep interviewing until patterns emerge
- Listening to users will help you list countless needs, pain point and desires
- Looks for gaps between expectations and how the world works
- How do you know which ones to action? Take inventory of the opportunity space
- helps you find the best path to the desired outcome
- Gives you structure to evaluate things
- Most product teams bounce from one opportunity to the next. Our job is not to address every opportunity
- It’s to find and address those that drive the desired outcome
- This is how we create value for for customer
- Focusing and limiting work to opportunities that might drive our desired outcome is what ensures that our products are viable
- The Aim is to address the customer opportunities that will have the biggest impact on our outcome first
- Start by taking an inventory of the possibilities - explore options
- Compare and contrast impact of addressing one opportunity against the impact of addressing another
- Rephrase opportunities that aren’t specific enough
- Group similar opportunities together
- Requires critical thinking - help to ensure that we are always addressing the most impactful opportunity.
The Opportunity Solution Tree
- Tree’s can tame backlogs.
- A flat list of opportunities is hard to prioritise. Some are interrelated, some are a subset of others. Some don’t come with clear solutions - some do. Much of that nuance is lost in a flat list
- The tree helps us expose and understand the complexity of the opportunity space
- A tree shows ‘Parent-child’ and ‘Sibling’ relationships
- We might ask … what are some other reasons why customers say “I can’t find anything to watch”
- Parent Problem - I can’t find anything to watch
- Child problem - I’m out of episodes of my favourite shows
- Child problem - the show I was watching was no longer available
- Child problem - I can’t figure out how to search for a specific show
- Helps us group customer needs - deconstruct the problem
- Breaks big opportunities into a series of smaller ones
- Allows us to tackle problems that seem unsolvable
- Allows us to deliver value over time
- Start by choosing easy things with large impact. Focus on shipping value over time.
- Solve enough smaller opportunities and you’ll have solved the larger one
- The tree helps asses and prioritise opportunities and make fast decisions
- The idea of distinctness is key - each opportunity has to be unique and self contained
- What customers do to address their needs today helps inform user research and helps give structure to the opportunity space
- Your user research can help you
- Deciding to watch something
- Choosing something to watch
- Watching something
- The end of the watching experience
- Review interview snapshots for opportunities to add
- Frame opportunities as a customer need or pain-point (not a solution)
- Make sure it’s common across customers - not unique to the one you spoke to
- It must feasibly affect the outcome if you address it
- If it feels like it can live under one or more branches...reframe it to be more specific OR split it into multiple distinct opportunities
- Example: “the interface is hard to use” ❌ This is too vague!
- I can’t figure out how to find a show I have in mind ✅
- I don’t like typing out the show name ✅
- Group similar opportunities together - and nest them under a new parent opportunity (don’t fret if your parent label wasn’t mentioned in an interview)
- Or combine them into one - if they are actually the same thing
- As you group siblings together, you end up with mini trees.
- Opportunity mapping looks deceptively simple - but it requires critical thinking
- Examine each opportunity to check its framed well, it’s clear, and can move the desired outcome
- Expect to spend more than 30 minutes, but don’t spend more than a day
- Get enough structure to the picture - but not so much that you’re overwhelmed by detail
- Structure is done, undone and redone.
- revise your tree often
- reframe opportunities as you learn more
- do just enough to capture what you already know - trust that it will continue to grow and evolve overtime
- BAD: framing opportunities from your companies perspective (frame as the customer instead)
- BAD: vertical opportunities = a parent with one child, who has one child in a single stack - collapse or add siblings
- BAD: opportunities have multiple parent opportunities - if top level opportunities represent distinct moments in time - then no opportunity should have two parents
- BAD: opportunities are not specific
- BAD: opportunities are solutions in disguise - distinguish an opportunity, by asking is there more than one way to solve this problem? if not → it’s an opportunity
- BAD: capturing feelings as opportunities - look for the cause of the feeling
Prioritising Opportunities, Not Solutions
You are never one feature away from success - and you never will be.
- We focus too much on the solution space - on shipping the next release. We assume that success comes from launching features - this is the build trap.
- Product strategy doesn’t happen in the solution space - it happens in the opportunity space. A solution-first mindset is good at producing outputs not outcomes.
- Customers don’t care about the majority of our releases. They care about their needs, pain points and desires
- Product strategy emerges from the decisions we make about... which outcomes to pursue, which customers to serve and which opportunities to address.
- Sadly many teams jump past this, and start prioritising features
Opportunity Space - customer pain points, needs and desires
Solution Space - experiments, solutions, features
Deconstruct large opportunities into smaller solvable ones. Adopt an agile mindset. Work iteratively and deliver value over time. Limit work in progress and explore multiple solutions.
Focus on assessing opportunities at the top level (or highest level possible) saves time comparing everything. Compare and contrast each set of parent opportunities against each other. Once you’ve picked the highest impact opportunity, you can ignore the others. Move down the levels until you have an opportunity with no children (A leaf node).
- How many (customers)?
- How often?
- Which of these sibling opportunities affects the most customers?
- How might addressing the opportunity influence your place in the market? or your position vs competitors?
- What external trends might (opportunities and threats) might impact which opportunity you choose?
- Table stakes - or differentiators?
- A missing table stake - could be bad news
- A differentiator - could open up new customer segments
- Consider organisational context - assessing and prioritising opportunities
- Support company vision, mission and strategic objectives
- Deprioritise what doesn’t fit with company values
- Political capital required for more controversial things
- Original thought - also control and influence of the team to get something done
- Evaluate how important each opportunity is for our customers
- How important are these (needs, pain points desires) to customers?
- How satisfied are they with the current solution?
- Don’t quantify everything (1-5 etc). That’s treating an ill-structured problem as if it were well structured - and will lead you to believe there’s one right answer. There isn’t.
- Balance having confidence in what you do - and leaving room for doubt in case you are wrong
- Make data-informed, subjective comparison for each set of features
- Balance data-informed vs analysis paralysis. You learn more from doing and observing that from thinking your way to perfect decisions
- Jeff Bezos on reversibility. Level 1: Hard to Reverse: Be cautious. Level 2: Easy to Reverse: Move fast, try and learn.
- Also known as one-way door or two-way door decisions
- Prioritising an opportunity is reversible. You’re making a commitment of the next few days or weeks. You’re not committing to addressing that opportunity.
- Discovery decisions are reversible decisions. Thinking of them as such frees us from confirmation bias.
- Delaying decisions until there is more data
- Over-relying on one set of factors as the cost of others
- Working backwards from your expected conclusion
Discovering Solutions - Solution Space
Idea Generation and Selection
- Once you know your opportunity (pain point, need or desire) where do you start?
- Quantity leads to quality
- Our first idea is rarely our best
- Creativity measured by
- fluency - the number of ideas we generate
- flexibility - how diverse the ideas are
- originality - how novel an idea is
- Originality and flexibility are correlated with fluency - volume of ideas helps everything
- the most original ideas tend to be generated toward the end of the session
- Product teams have backlogs full of ‘first ideas’. AND even though we have a huge number ideas, we don’t have many for each opportunity. We aren’t going deep enough where it matters.
- Innovate where there’s a strategic opportunity (generate several ideas to ensure you uncover the best ones) don’t reinvent the forgotten password flow
- Four rules of brainstorming (from the father of brain storming Osborn)
- Focus on quantity
- Defer judgement
- Welcome unusual ideas
- Combine and improve ideas
- more ideas, more diverse, and more original
- People work harder when working individually
- Group conformity challenges
- Production blocking - interrupting each others thinking
- downward norm setting - group performance tends to be limited to the lowest-performing member
- Cast aside limiting beliefs - you aren’t bad at generating ideas
- Take frequent breaks - spread it out throughout the day
- Change scenery
- Experiment - find what works best for you
- Take advantage of incubation - taking a break or sleeping on it
- Look for analogous products for inspiration - competitive research
- Start with competition then look wider - what industries have solved similar problems? look for similarities in the target opportunity
- What might extreme users need?
- First time users vs power users vs disabilities vs young vs old
- Consider wild ideas - wild ideas can improve more feasible ideas
Putting it all into practice
- Review your target opportunity - context - make it distinct
- Generate ideas alone
- Share ideas across your team
- Repeat steps 2 and 3 until you have 15-20 ideas for the target opportunity
- Remove ideas that don’t solve the target opportunity (this happens!)
- Dot-vote as a team to get down to three ideas (how well does the idea address the target opportunity?)
- Everyone gets 3 votes - Can spend them how they like
- Do multiple rounds if needed:
- Remove the winner
- Get people to go again
- Go until you have 3 left
- Not including diverse perspectives
- Generating too many variations of the same idea
- Limiting ideation to one session
- Selecting ideas that don’t address the target opportunity
Identifying Hidden Assumptions
- Confirmation Bias: Seeking out evidence that confirms our beliefs
- Confirmation bias escalates with investment. The more we invest, the more committed we become
- explore why your ideas won’t work
- working with a set of ideas!
- There isn’t enough time to A/B test or prototype everything
- INSPIRED - the best product teams complete a dozen or more discovery iterations per week
- This is only possible if you test ASSUMPTIONs - NOT ideas
- Assumption testing is quicker than idea testing -
- pace helps us with ‘escalation of commitment bias
- Knowing what assumptions you’re making is the biggest barrier to testing them
Types of Assumptions
Does anyone want it? Will anyone get value from it?
Should we build it? Many ideas work for customers but not the business. Ideas need to generate more revenue than they cost to build, service and maintain
Can we build it? Technical, Legal, Security, Culture, Regulations
Is it usable? Is it discoverable? Will they find it? Will they understand how to use it or what they need to do? Are they able to do what we need them to do? Is it accessible?
Is there any potential harm in building this idea> What data are we collecting? How are we storing it? How are we using it? If our customers had full transparency, would they be OK with it?
- Start by assuming the solution already exists (and that you’re mapping out what end-users will do to get their value from the solution)
- Identify the key actors - who needs to interact with who for it to work?
- Map out the steps each actor has to take for anyone can get value from the solution
- Sequence the steps horizontally over time (if steps are optional list them as such)
- Put actors on rows
- Every time you assume that an end-user will do something, you are making assumptions of different types (desirable, viable, usable, feasible, ethical) - you may have a few for each
- Example - Our subscriber comes to our platform to watch sports
- Desirability: Our subscriber wants to watch sports.
- Desirability: Our subscriber wants to watch sports on our platform.
- Usability: Our subscriber knows they can watch sports on our platform.
- Usability: Our subscriber thinks of our platform when it’s time to watch sports.
- Feasibility: Our platform is available when our subscriber wants to watch sports.
- Imagine its six months into the future, you launched, and it failed. What went wrong?
- Phrase question as if the outcome is certain.
- You likely made assumptions on your tree - walk back up to the top and capture assumptions
- Generate assumptions by asking the following
- this solution will address the target opportunity because ...
- addressing the target opportunity will drive the desired outcome because ...
- You might need to test, that your product outcome is going to drive your business outcome
- Data, addiction, accessibility, inequality, privacy, trolls, user-user harm, brand, customer expectations, opportunity cost
- Front Page Test
- Use a 2x2
- How much do we know? (about this assumption)
- How important? (is this assumption)
- Test Leap of faith assumptions
- Make the mapping relative to other assumptions
- do we have stronger or weaker evidence?
- more or less important than x assumption?
- Not generating enough assumptions
- Phrasing assumptions such that you need them to be false (positive framing is easier to test. Customers will remember their passwords)
- Not being specific enough
- Favouring one category at the cost of others
Testing Assumptions, Not Ideas
- Gather evidence and test assumptions for your top 3 ideas at the same time - this helps us not fall in love with one idea
Our goal is to collect more evidence on the assumption:
- Collect data about what people do in context (not what they think or say they do)
- Simulating an experience is a strong test - we hope the participant will behave as in the situation we emulate
- Choose the moment to simulate carefully
- Account for shortcomings of the simulation when evaluating results
- Set specific with your evaluation criteria (or hurdle rates in advance)
- If our assumption is true, what would we expect the participant to do?
- Log how many do, and how many don’t
- helps you align as a team
- helps you guard against confirmation bias
- You can’t prove it works - the burden of truth is too high. If your assumption holds true - you’ve reduced risk though.
- Key outcome is to agree as a team on the smallest assumption test you can design that still gets you results that the team feel comfortable acting on
- We don’t want to invest time, energy and effort into an experiment if we don’t have an early signal we’re on the right track
- Rather than starting with a large scale experiment or A/B test - you’ll be surprised how much you can learn from getting feedback from a few customers
- Each round of testing helps you inform what’s the next most important assumption to test
- You stop testing when you feel comfortable
- you want a representative sample - the smaller the sample, the more chance you’ll get a false signal by selecting one type of person too much
- false negatives shouldn’t matter, if your testing your assumption in a few ways
- use a mix of research methods to better understand the assumption your testing
- False negatives are often not that bad, there are always more ideas and we can always try again
- Either move to test another assumption OR move to a bigger experiment
It’s not science, our goal is to mitigate risk - not to seek truth. Our bar can be lower
You need to get better at quickly executing assumption tests
Tools to help:
Unmoderated user testing
Post a stimulus (prototype) - Define tasks to complete and questions to answer. Participants come in their own time. You get a video of their work. - careful with screening - you want your own users - you can even upload your own audience
One question surveys
When was the last time you watched sport? Please select all the sports you watched in the last month?
How many people have searched for sports. Or related to sports
- See UX for Lean Startups - for a longer list of methods
- overly complex simulations
- using %s instead of numbers when defining evaluation criterial
- not defining enough evaluation criteria
- testing with the wrong audience
- designing for less than the best-case scenario
- design your assumption tests so they are likely to pass
- if they still fail - then that’s a sign /red
Measuring the impact
- Don’t get distracted by your assumption tests - stay true to solving your outcome
- Discovery and delivery are iterative - you’re never done with discovery
- Iteratively invest in experiments. Start small and grow your investment over time.
- As experiments grow, you’re going to need to test them with a real audience
- Instrument what you need to evaluate your assumption tests
- Define evaluation criteria for each assumption - measure it
- Count #people who do an action (not #actions) if one person doing many actions isn’t as valuable as many people doing actions
- On search results - measure the position in results - measure the ratio of views to conversions
- Measure what you need to evaluate progress toward your desired outcome
- If your product doesn’t go across the whole user journey - you might not know the end outcome you’re trying to influence
- incentivise people to tell you what happened - survey them
- JUST BECAUSE SOMETHING DOESN’T HAPPEN ON YOUR PLATFORM, DOESN’T MEAN YOU SHOULDN’T MEASURE IT
- Don’t be afraid to measure hard things
- Don’t try to measure everything at the start - learn as you go
- You don’t want to wait to have the final answer before iterating - find leading measures
- Run a small test on one day - see if you move your metric that day - for those that engage
- Getting stuck trying to measure everything
- Hyper-focusing on your assumption tests and forgetting to walk the lines of your opportunity solution tree
- Forgetting to test the connection between your product outcome and your business outcome
Managing the cycles
- Continuous discovery isn’t sequential. It’s often circular and more messy.
- The fruit of discovery work is often the time we save when we decide not to build something
- Sometimes customers want something - but don’t expect your brand to solve that problem for them
- Sometimes you’ve got a need that can be solved, but nows not a good time, so you can push it back on the roadmap
- Balance short with longterm - low hanging fruit, with harder long term problems
- You can use your discovery work and quick win tests - to make the case to go after harder solutions
- Iterate through small opportunities for big impact
- Overcommitting to an opportunity
- Avoiding hard opportunities - don’t confuse quick testing and iterative delivery with avoiding hard solutions - the longer we spend building - the more we should de-risk
- Drawing conclusions from shallow learnings
- Giving up before small changes have time to add up
Show Your Work
The more leaders can understand where teams are, the more they will step back and let teams execute
- Get your stakeholders on board - else everything else is wasted
- stakeholders have thoughts on solutions, not grounded in discovery
- the hippo is going to win
- Don’t start with conclusions
- Show your opportunity solution tree
- Start at the top
- Remind stakeholders what the desired outcome it
- Highlight the top-level opportunities (you can invite them to add more opportunities)
- Share how you assessed the priority of the space
- Share the context about your target opportunity - customer pain point
- use the snapshots to back that up
- Share the solutions you generated
- Share the set of 3 solutions you plan to move forward with (ask them if they’d have chosen different)
- Share the assumptions and how you prioritised them
- Share assumption tests - and data - repeat
- Continuously manage your stakeholders
- Showing your visual decision making progress takes your stakeholders along for the journey
- Don’t present a conclusion, show your process, which leads to more buy-in and long-term success
- Telling instead of showing
- Overwhelming stakeholders with all the messy details
- Arguing with stakeholders about why their ideas won’t work
- Trying to win the ideological battle instead of focusing on the decision at hand
Developing Your Continuous Discovery Habits
Start Small and Iterate
- I rarely had the support of senior leadership to do product discovery well.
- Find a way to get closer to the customer - find a way to advocate for human-centred design.
- I had a strong sense of agency - you choose how you work - you might not be able to change the way companies work.
- Instead of asking for permission - start small and iterate from there
- Build your trio - don’t work alone - build the relationships yourself - start small with them
- Start talking to customers - it’s the keystone habit - weekly changes everything - continuous interviewing
- Work Backward - As you work on requirements for the solutions you were asked to build, story map your ideas - use them to identify hidden assumptions - be aware of your assumptions
- Identify assumptions with stakeholders - the idea will improve right then and there
- Document what impact people think a feature will have
- Do post-release impact reviews (expected impact vs actual impact)
- Are we trying to solve a customer problem with this feature?
- The best time to advocate for discovery is when a feature falls short of expectations
- Never say ‘I told you so’ - be a collaborative problem solver
- Use retrospectives to reflect and improve
- What did we lean in this period that surprised us
- How could we have learned sooner?
- Focusing on why a strategy won’t work - instead of focusing on what is within your control
- Being the annoying champion for the right way of working
- Waiting for permission instead of starting with what is within your control
If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions Albert Einstein
Managers must convert society’s needs into opportunities for profitable business Peter Drucker
Too often we have many competing goals, that all seem equally important Christina Wodtke - Radical Focus
People don’t know what you want until you show it to them. Our task its to read things that are not yet on the page Steve Jobs