Product #38

Hacking Growth by Sean Ellis and Morgan Brown (2017)

This book is a practical playbook for establishing and operating a growth team, emphasising the importance of rapid experimentation in product. It provides valuable insights into the significance of Acquisition, Activation, and Retention. It is a concise, insightful, and actionable guide that offers everything you need to get started.

Key Highlights

• You can unlock rapid growth and get more from your marketing spend if you breakdown the traditional silos of marketing and product and setup teams to focus on acquisition or retention. Dropbox acquired 1/3 of customers through offering free storage for referrals.
• A multi-disciplinary team with a mandate to find growth potential is powerful. Focus them on continuous testing and tweaking of the product (it’s features, messaging, and its acquisition, retention and activation methods). Give them exec sponsorship and point them at acquisition, activation or retention.
• Test if you have product-market fit before spending a lot on marketing. To determine this, ask your customers how they would feel if they could no longer use your product. If 40% or more express being very disappointed, it indicates you’re onto something.
• Tag your product so you can understand what's happening from the beginning to the end of the funnel. Data tells you what users are doing - not why they are doing it. You'll need to conduct some user surveys or interviews to figure out what's going on.
• The 'Aha! moment' is when users experience the value of a product for the first time.
• Once you've identified the conditions that make the ‘Aha! moment’, turn your attention to getting more customers to experience that moment as quickly as possible. Spend about 30% of your time and effort on this.
• Your essential metrics are determined by identifying the actions that correlate most directly to users experiencing the core value of your product.
• Create your growth equation - its simplicity helps focus. It should include each of the steps users must take to reach the aha moment (and how often they take them).
• Hone your growth equation and narrow your focus by choosing a single metric of success that all growth activity is geared towards.
• Experiment and test at a high tempo. The faster you learn, the faster you can grow. Most experiments fail to produce results, so volume is key. Set a weekly heartbeat to encourage experiment velocity, discuss results, and decide what to test next. Think of a minimum viable test. Scattershot experimentation is a waste of time and effort - instead, focus on growth levers. Set a minimum number of tests per week.
• Don’t be afraid to double down - push more and more on successful levers. Push past local maximums by taking moonshots. Expect most of the gains to come from more modest changes.
• The growth hacking cycle: Data analysis → insight gathering → idea generation → experiment prioritization → running the experiments → review results → decide
• Small changes in language and messaging can have a significant impact on acquisition. For example, changing 'Store your photos online' to 'Share your photos online', or 'Find a date' to 'Help people find a date'.
• Channel Strategy - Research and prioritize your channels. Find one or two that have high potential - optimize them for cost-effectiveness and reach. Consider the needs of your business model and the characteristics of your customers. Fish where they are.
• Rank each channel based on cost, targeting, control, input time, output time, and scale.
• Virality = Payload x Conversion Rate x Frequency
• Create a funnel report - then conduct qualitative research to understand what's happening at each stage. "What's the one thing that nearly stopped you from completing your order?"
• The Compounding Value of Retention: Retaining a customer for a longer period of time increases the opportunity to earn revenue from them. By increasing the lifetime value of a customer, you can allocate more resources towards growth and make better revenue predictions. Furthermore, longer customer retention allows you to gain a deeper understanding of their needs and desires, enabling better personalization and ultimately, higher earnings.
• Providing a product that addresses the needs of, or delights of a customer is the best way to drive retention. Better retention will drive better results from viral marketing - the longer they stay, the more likely they’ll talk to others about it.
• Success of Trigger = Motivation to take action x Ease of taking the action
• Behaviour = motivation x ability x trigger
• A growing retention rate is a strong indicator of a successful product. This is often attributed to stored value - the more you use the product, the more valuable the data becomes, increasing the likelihood of continued usage.
• The Hook Model: Trigger → Action → Reward → Investment (repeat)
• Ongoing Onboarding: Continuing to educate your customers about the value they can receive from your product. Getting users to a place where they’re getting the most value out of the product is called ramp up.
• Monetisation is about earning more revenue from each customer over time - increasing LTV (lifetime value). Look at revenue by cohort. Find your high profit vs low profit customers. Look at average revenue per users.
• Potential Cohorts: Group by: age, location, gender, types of item purchased, features used, acquisition source, type of device, type of browser, number of visits, date of first purchase
• Look for patterns in retention rates - correlations will give ideas for experiments.
• Harness the power of simple recommendations by using the Jaccard similarity coefficient. This coefficient measures the similarity between two items (A and B) as the size of the intersection of A and B divided by the union of A and B. The intersection represents the number of people who purchased both products, while the union represents the number of people who bought either product independently.
• Use the 4 question pricing survey to find the optimal price (Van Westendorp Price Sensitivity Meter · Link)
• Compare personas. Do they value the same features? Do they have the same willingness to pay? What is the cost of acquisition? What is the Lifetime Value?
• Charge based on value. Find a way to charge customers more if they receive more value. Does your value metric align with the customer's perception of value? Does it scale as the customer uses the product more? Is it easy to understand?

Full Book Summary · Purchase on Amazon · More Book Summaries

Subscribe and share Product so you and your team can keep pace
with Product Management and related fields.
< Subscribe Now Button >

In the News

• Meta is developing open source AGI · Article
• A Guided Tour of Apple Vision Pro · Video
• Intercom Customer Service Trends Report 2024 · Report
• Stack Overflow Developer Survey · Report
• A survey of 2778 AI researchers, includes AGI probability horizon curve · Report

Quick Links

• Richard Rumel is on Lenny’s Podcast this week · Podcast
• Five ways to build a $100 million business · Article
• The protocol force · Article
• H3 a hexagonal-shaped hierarchical geospatial indexing system · Article
• Slack CPO on how to take bigger, bolder product bets · Article

Seven Failure Points When Engineering a Retrieval Augmented Generation System (2024)

Scott Barnett, Stefanus Kurniawan, Srikanth Thudumu, Zach Brannelly, Mohamed Abdelrazek

“Software engineers are increasingly adding semantic search capabilities to applications using a strategy known as Retrieval Augmented Generation (RAG). A RAG system involves finding documents that semantically match a query and then passing the documents to a large language model (LLM) such as ChatGPT to extract the right answer using an LLM. RAG systems aim to: a) reduce the problem of hallucinated responses from LLMs, b) link sources/references to generated responses, and c) remove the need for annotating documents with meta-data. However, RAG systems suffer from limitations inherent to information retrieval systems and from reliance on LLMs. In this paper, we present an experience report on the failure points of RAG systems from three case studies from separate domains: research, education, and biomedical.”

The paper focuses on Retrieval Augmented Generation (RAG) systems, combining large language models (LLMs) with semantic search capabilities. RAG systems are designed to overcome limitations of LLMs by retrieving documents that semantically match a query for generating responses.

Validation of RAG systems is feasible only during operation.The robustness of RAG systems evolves over time rather than being fully designed at the start.

View the Paper

Book Highlights

Insert Artwork

Occasionally, having a stakeholder insist on a change can lead you down a path to improving the app in a way that you never expected, solving problems you might not have uncovered otherwis

Tom Greever · Articulating Design Decisions

It begins with a clear hypothesis that makes predictions about what is supposed to happen.

Eric Ries · The Lean Startup

Great product leaders spend the majority of their time focusing on this discovery process.

Richard Banfield, Martin Eriksson, Nate Walkingshaw · Product Leadership

It still took many years for the iPhone to come to maturity, but imagine if Apple had released an iPhone with apps, location awareness, multitasking, and a large screen all at once? The price point would have been outrageous, the device completely alien, and the smartphone almost certainly a disastrous failure. Amber Case · Calm Technology

What I’m Listening to

High Speed · Nicolas · Elements · 3 Lights

Opportunities multiply as they are seized.” Sun Tzu

In the next issue….

Write something about the next issue to tease people about what’s included,