Product #50

Accelerate · Author · 2018

There’s no shortage of books putting forward an opinion about how to build digital products - few though are evidence based. The authors behind Accelerate have based their findings on research and data. As such, this is book has become incredibly influential in product and engineering circles. The book provides an introduction to DevOps but it’s also one of the best books on building high performing teams. It’s now best practice across the industry to track the DORA metrics. There’s very little cultural advice in this book I disagree with.

Key Ideas

Companies should look to measure and improve capabilities and avoid 'maturity models'. The authors' research links software development performance to organisational performance. Adopting DevOps best practices can dramatically increase frequency of deployments, reliability, recovery time, and speed from keyboard to production. Studies have shown you don't have to trade speed for quality; you can have both. Avoid measuring and rewarding developers for throughput and operations for stability, as it's going to end in tears. Successful measures need to focus on global outcomes, not local or output.

Introducing the DORA metrics - every team should be measuring these:

1. Deployment Frequency (Tempo): Using deployment lead time as a proxy for batch size. Reducing batch size reduces cycle time, flow variability, accelerates feedback, reduces risk, improves efficiency, increases motivation, and reduces cost. How often do you deploy to production or to the app store? Options: On-demand, hourly, daily, weekly, monthly, 6 monthly, fewer
2. Lead Time for Changes (Tempo): Shorter is better. Faster feedback, quicker course correction. Better in an outage. Two important periods, but only one is measured. Time to design and validate (hard to know when to start the clock). Time to build and deliver (easier to measure). Time from code committed to code deployed? Options: less than 1 hour, 1 day, 1 week, 1 month, 6 months, longer
3. Mean Time to Recovery (Stability): Failure is inevitable, so we ask how quickly can service be restored. How long does it take to restore service when there's an unplanned outage? Options: less than 1 hour, 1 day, 1 week, 1 month, 6 months, longer
4. Change Failure Rate (Stability): What % of changes to production fail? Including infrastructure and configuration changes result in degraded service or require remediation?

There is no tradeoff between speed and quality. High performers do better at all the measures. Software performance is a good predictor of company performance (profit, market share and productivity), hitting goals, customer satisfaction, etc. Companies should distinguish which technology is strategic and which isn't, and acquire non-strategic services as SaaS.

Culture is important, but people think it's intangible and hard to measure. Organisational culture predicts the way that information flows through the organisation. Good information flow provides answers, is timely, and presented in a way it can be effectively used. Better information flow leads to better performance:

1. Trust and cooperation between people across the organisation
2. Better organisational higher quality decision making
3. Likely to do a better job with people; problems are rapidly discovered and addressed

Google found that team dynamics are more important than the individuals within a team. How failure is dealt with is a good measure. You need psychological safety.

Continuous delivery is a set of capabilities that allow us to get changes into production safely, quickly, and reliably. Key principles of continuous delivery:

1. Build quality in (so less pressure on testing)
2. Detect issues quickly, so they're cheap and easy to resolve
3. Work in small batches
4. Computers perform repetitive work tasks, people solve problems
5. Continuous improvement
6. Everyone is responsible (system level outcomes)

To implement continuous delivery you need comprehensive configuration management (fully automated version control), continuous integration, and continuous testing all the way through building. The impact of continuous delivery includes decreased deployment pain and employee burnout, stronger identification with the organisation, higher levels of software delivery performance, sustainable development, and time saved on rework (bugs and patches, as a proxy of quality).

Reduce dependencies across teams. Architecture should map to teams. Let teams choose their own tools. Focus on engineers and outcomes, not tools and technologies. Design things so they have freedom. Build security into software development. Shift left with security - review features as soon as possible. Security should be involved in design. The Rugged movement advocates refusing to be vulnerable, choosing to be rugged, and making being rugged everyone's responsibility.

Lean management practices involve removing work in progress, making metrics and dashboards visible (to engineers and leaders) with ease of access being key, and using these tools to make decisions on a daily basis. Approval by an external body slows things down and doesn't improve reliability. Segregation of duties + peer review + auditable pipeline is great.

Companies still spend too long on requirements, budgeting, large releases, and customer feedback seems to be an afterthought. Companies should test product design and business models by frequent user testing. Adopt a Lean Startup experimental approach to product development, using lean and design thinking. Pivot early and often; take an experimental approach.

Lean product development capabilities include:

• Small releases (1 week max, MVP)
• Flow of work from business to customers, made visible to everyone
• Actively and regularly seeking customer feedback and incorporating it into products
• Development team having authority to change specifications as part of the process without approval

Software performance predicts lean product development practices, creating a virtuous cycle. Working in small batches makes rapid development possible, with short lead times and faster feedback loops enabling A/B testing. Team experimentation involves seeking input from customers throughout the process. The ability of teams to try out new ideas without approval is important.

To decrease burnout:

• Create a blame-free environment
• Make deployments painless
• Ensure effectiveness of leaders
• Invest in DevOps
• Support experimentation and learning
• Give people some time to work on new and exciting projects
• Align organizational values with individual values

Hire, retain, and delight employees. Higher Employee NPS leads to higher retention. Diversity matters for smarter, better team performance and business outcomes. Inclusion must be present for diversity to take hold.

Leadership is about inspiring and motivating those around you. Transformation leader characteristics include vision, inspirational communication, intellectual stimulation, supportive leadership, and personal recognition. Transformational leadership involves inspiring, empowering, developing, identifying, engaging, and aligning. It is statistically significant and highly correlated with NPS.

Enabling practices include continuous experimentation and learning. Culture should involve a climate of learning (space to explore ideas), being safe to fail, having space to share knowledge, letting people choose their tools, and making monitoring a priority.

For organisational transformation:

• Visualise priorities and objectives as they progress
• Keep problems visualised until they are solved
• Use a matrix structure with value streams and squads; squads have product owners, are cross-functional, and follow the 2 pizza rule (biz dev ops)
• Visualise work in progress
• Hold stand-ups but less meetings overall
• Let people own quality; let them not release if they're not comfortable
• Standard ways of working save time and energy

How do we learn how to learn? Coaches question assumptions and challenge behaviours. Do it in cycles so it becomes routine. Get people to share how they work. Develop and maintain the right mindset.

Do things on your own - don't contract out to a consulting firm to transform your organisation. Demonstrate behaviours; don't delegate them.

Full Book Summary · Amazon

Subscribe Button 

Quick Links

A PM’s guide to wireframes · Article

The 4 Levels of PMF · Article

The illusion of productivity and busyness matrix · Article

Building reliable systems out of unreliable agents · Article

Relax PMs, we're ready for AI · Article

8 UX Tips for Designing AI Products · Article

What’s your scientific problem solving method of choice? · PDCA · DMAIC · 8D · BML

Maps of Bounded Rationality · Daniel Kahneman · 2003

Abstract

Herbert A. Simon (1955, 1979) had proposed much earlier that decision makers should be viewed as boundedly rational, and had offered a model in which utility maximization was replaced by satisficing. Our research attempted to obtain a map of bounded rationality, by exploring the systematic biases that separate the beliefs that people have and the choices they make from the optimal beliefs and choices assumed in rational-agent models. The rational-agent model was our starting point and the main source of our null hypotheses, but Tversky and I viewed our research primarily as a contribution to psychology, with a possible contribution to economics as a secondary benefit. We were drawn into the interdisciplinary conversation by economists who hoped that psychology could be a useful source of assumptions for economic theorizing, and indirectly a source of hypotheses for economic research.

Key Points:

Renowned psychologists Daniel Kahneman and Amos Tversky revolutionized our understanding of human decision making by proposing that people have two distinct systems of thinking. "System 1" is fast, automatic, and intuitive, allowing us to make quick judgments and react instinctively to our environment. In contrast, "System 2" is slower, more effortful, and controlled, enabling us to engage in deliberate reasoning and problem-solving.

While both systems play important roles, people often rely heavily on the rapid, intuitive judgments of System 1. This can lead to the use of mental shortcuts known as heuristics, such as judging the likelihood of an event based on how easily examples come to mind (the availability heuristic) or estimating unknown quantities by anchoring on a familiar reference point and adjusting insufficiently (the anchoring effect). Although heuristics can be efficient, overreliance on them renders people susceptible to predictable errors and cognitive biases.

The way options are framed or presented also exerts a powerful influence on decision making. Equivalent outcomes can be perceived very differently depending on whether they are framed as gains or losses relative to a reference point. Due to loss aversion, people tend to strongly prefer avoiding losses to acquiring equivalent gains. This reference dependence violates the rational principle that decisions should be based on final states rather than changes.

People's choices are also distorted by biases in predicting their future preferences and well-being. For instance, they often exhibit duration neglect, basing evaluations disproportionately on the peak and end of an experience while neglecting its duration. Attempts to forecast future utility are susceptible to biases like focalism, causing people to overestimate the impact that specific events will have on their happiness.

In the domain of risky choice, Kahneman and Tversky developed Prospect Theory as a descriptive model of decision making that better accounted for observed behavior than the normative Expected Utility Theory. Prospect Theory incorporates psychological principles like reference dependence, loss aversion, and nonlinear probability weighting to explain phenomena such as the fourfold pattern of risk attitudes.

The profound implications of these psychological insights extend beyond the laboratory to real-world domains like economics, policy, and consumer choice. By recognizing the ways in which human judgment and decision making deviate from rational models, practitioners can design more effective interventions, products, and institutions. Behavioral economics has emerged as an important field incorporating these psychological findings to improve both descriptive validity and prescriptive recommendations. Continued integration of psychological research with economic analysis promises to yield more accurate models of behavior and enhance human welfare and wellbeing.

View the Paper

Book Highlights

We value the things we make together. We’re going to set our Objectives together. Christina Wodtke · Radical Focus

Despite exciting progress in few-shot learning research, most ML algorithms still require many examples to learn a pattern. When a task is repetitive, each pattern is repeated multiple times, which makes it easier for machines to learn it. Chip Huyen · Designing Machine Learning Systems

A product that doesn't have desirability designed into it might address a robust market need, but any success it enjoys will be the success of the dancing bear. The single greatest weakness of dancing bearware is that it never generates customer loyalty. Without the long-term brand loyalty of satisfied customers, your entire company is highly vulnerable to competition. Alan Cooper · The Inmates Are Running the Asylum

What have those companies done to access instant data? First, they've changed the way they manage themselves, their teams, and their companies; they've changed how they run meetings, how they make decisions, and how they collaborate. Employees are data literate: They understand how to access the data they need, how to analyze it, and how to communicate it well. Tomasz Tunguz, Frank Bien · Winning With Data