Review

I'm not sure if the 'Flow Framework' is the golden goose of transformation. I found the theory intriguing, but I have reservations about implementing it within a large organisation. There was a lack of practical implementation advice, and based on my experience, I believe it would be challenging to implement without a team of consultants. Nevertheless, there are plenty of interesting nuggets and principles in this book. It has certainly influenced my perspective on team topologies, and managing those zero-sum tradeoffs between features, quality and risk.

You Might Also Like:

Team Topologies

This book is essential reading if you need or want to learn a little more about organisational design. The model in this book is a simplification of a messier reality, but it’s a useful frame of reference. It establishes a limited number of team types and interaction types between teams. The authors then explain how fracture planes and sensing/triggers can help you identify the best team topologies.

andrewclark.co.uk

Book Review and Summary: The Mythical Man Month by Fred Brooks

Get a comprehensive review and summary of the classic book "The Mythical Man Month" by Fred Brooks, the software engineering and management expert. Learn about the key principles and practices for managing software development projects and teams effectively.

andrewclark.co.uk

Book Summary: Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim

Get a comprehensive review of the best-selling book "Accelerate" by Nicole Forsgren, Jez Humble, and Gene Kim. Discover the key insights and takeaways from this must-read for tech leaders and professionals

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

• IT and digital transformation failures often stem from a business-technology disconnect. This disconnect manifests in a project focus over product focus, a cost focus over profit focus and a timeline focus over delivering business value.
• Adopt a product-oriented approach to software delivery. Prioritise delivering business value over completing technical tasks. Embrace the Flow Framework for managing software delivery.
• The Flow Framework bridges the gap between business and technology. It focuses on measuring and managing the Value Stream Network: a critical layer for successful software delivery at scale.
• Prerequisites for success include:
• A managerial culture aligned with product mindset.
• An understanding of the rapidly changing market.
• An ability to bring software-based products to market and adapt.
• Defining how to measure business value in software delivery is crucial. The Flow Framework provides a way to achieve this.

• The three epiphanies
1. Productivity declines and waste increases as software scales due to disconnects between the architecture and the value stream.
2. Disconnected software value streams are the bottleneck to software productivity at scale. These value stream disconnects are caused by the misapplication of the project management model.
3. Software value streams are not linear manufacturing processes but complex collaboration networks that need to be aligned to products.

• We’ve learnt a lot about software delivery, but we need to apply it to how businesses approach software projects. DevOps and Agile haven’t been adopted outside of technology teams. We need a new framework that bridges the gap between business and technology teams and enables the transition from project to product thinking.
• The framework must scale the ‘three ways of DevOps’ - flow, feedback, and continuous learning
• To avoid the pitfalls of local optimisation focus on the end-to-end value stream. If you manage a transformation according to cost alone, you will reduce productivity. Treating IT as a cost centre led to a disconnect, valuing cost-cutting over delivering value.
• Over-reliance on proxy metrics can mislead decision-making, overshadowing the need for metrics that truly reflect business outcomes. Jeff Bezos advocates resisting proxies, emphasising the importance of outcome-based metrics over process-based proxies.
• The project end fallacy overlooks software economics, notably technical debt, which can degrade software quality and add maintenance costs.
• Project-oriented management, where resources are allocated to specific projects, is based on the Taylorist assumption that people are interchangeable and replaceable. This approach is unsuitable for complex knowledge work like software development, where building expertise and relationships takes time.

• Software products and value streams need to be as visible as in manufacturing lines, but the amorphous nature of software products makes this more difficult. Data collection and visualisation tools are available, but the issue lies in creating effective abstractions for visualisation at the business level.
• DevOps teams have clarity on what to visualise, such as deployment metrics. Development teams use Agile methods like Scrum or kanban for visibility at the team and specialist levels. Flow metrics provide the needed visibility at the business level.
• Measuring flow metrics in real-time across all value streams is complex but necessary.

• Activity-oriented proxy metrics should be replaced with business outcome-oriented metrics.
• The Flow Framework connects business and technology, enabling an organisation-wide feedback loop to accelerate the flow of business value to customers and facilitate organisational learning.
• The Flow Framework bridges the gap between business and technology, enabling product-oriented software delivery. It is made of:
• Product-oriented mindset: Shift from project-centric approach to viewing software as evolving products.
• Value Stream Networks: Infrastructure providing automation and visibility for software delivery, similar to manufacturing production lines.
• Flow Items: Core elements tracked and optimised within the Flow Framework:
• Ideas: Capturing and prioritising business needs for software solutions.
• Features: Units of functionality delivering business value.
• Releases: Bundles of features delivered to customers.
• Deploys: Moving features from development to production environments.
• Everybody in your organisation should be comfortable answering:
• Who is the customer?
• What value is the customer pulling?
• What are the value streams?
• Where is the bottleneck?
• 5 principles of ‘Lean Thinking’:
1. Precisely specify value by specific product
2. Identify the value stream for each product
3. Make value flow without interruptions
4. Let the customer pull value from the producer
5. Pursue perfection

• Defining the business results from tracking flow is essential before proceeding.

• The Flow Framework aims to track business results for each value stream and correlate investments with outcome-based metrics.
• Transitioning from a project-oriented model to a product-oriented model is necessary to align value streams with products and IT portfolios.
• Canonical metrics for each value stream include measures of value, cost, quality, and happiness.
• Creating a value stream dashboard with common Value Stream Metrics can provide transparency and inform decision-making.
• Value Stream: the end to end set of activities performed to deliver value to a customer through a product or service
• Flow Unit: A unit of business value pulled by a stakeholder through a product’s value stream.
• MECE flow units in a software value stream:
• Features
• Defects
• Risks (security, regulatory, compliance)
• Debts (technical debt)
• Only debt that increases the flow through the value stream should be prioritised

• Flow distribution: allows the business to scale planning that product and engineering teams do on a daily basis. Thinking in flow distribution trade-offs elevates the understanding of the business trade-offs. You can make visible the pain of delaying defect work and pushing out features to meet a business deadline.
• Flow Velocity: a measure of the number of each flow item being completed over time. Helps understand how much business value is being delivered to the customer. Velocity is across all flow items. If 10 features, and 5 risks are completed in a release, the flow velocity is 15. It differs from the traditional agile velocity measure because it doesn’t rely on estimation. Flow velocity is the empirical measure of productivity based on direct observation of value streams, flow items are tied to real business value (as PdMs should be selecting the work that’s the most impactful). Flow items should have their value defined before they enter the stream.
• Flow Time: to determine the speed of delivery we need to measure flow time. There are 4 flow states: New, Waiting, Active, Done. Flow time = Active → Done. Flow time is a measure of how long it takes to deliver a flow item, from the decision to take on the work to the value being delivered to the customer. Lead time isn’t as useful in software as things can sit on the backlog for ages.
• Flow load: represents concurrent work-in-progress (WIP) within a value stream. It is the number of flow items being actively worked on in a value stream. .
• Flow Efficiency: Measures the active work time on flow items against the total time elapsed in the value stream. Flow efficiency drops with increased dependencies causing delays and bottlenecks.
• Defining flow items: Each flow item can be measured by observing the tool network. With the flow items, we can measure the flow of business value through the tool network and correlate that to business results.
• Flow Distribution: The proportion of each flow item within a value stream, adjusted depending on the needs of each stream to maximise business value. Setting flow distribution: This is crucial for the Flow Framework as all other metrics depend on it. We need to track the target flow distribution for each product's value stream to determine the type of business value delivered.
• Measuring flow velocity: We must measure the delivery rate of each flow item to the customer over time. This is flow velocity, adapted from Agile software development's concept of velocity.
• Tracking flow time: Flow distribution and flow velocity provide an empirical measure of the work done over time, but they don't show how quickly the work cycles through the system. Flow time defines the speed of delivering business value to the market.
• Measuring flow load: To optimise flow, we need to avoid overloading the value stream with excessive work in progress (WIP). The flow load metric tracks this at the value stream level (e.g., indicating the number of features worked on in parallel).
• Tracking flow efficiency: Within each product's value stream, flow items are either being worked on or waiting. Flow efficiency measures the ratio of productive to waiting time, helping fine-tune value streams for productivity.
• Connecting to business results: Value, cost, quality, and happiness are the four categories of business results to be tracked in the Flow Framework. This correlation between software investment and business performance provides a common set of results-oriented metrics to link the business with IT.
• You need to track each of these measures for each value stream:
• Value: measures benefit to the business produced (e.g. revenue, MRR, 7DAU).
• Cost: of the value stream to the business (cost of all staff, operations and infrastructure)
• Quality: of the product produced as from the customer perspective (escaped defects, NPS, retention.
• Happiness: engagement of the staff on the stream (eNPS, employee engagement).

• The Flow Framework hypothesis suggests that too much flow distribution allocated to features has resulted in excessive technical debt, impacting quality trade-off and recall rates.

• The Gemba walk is a fundamental part of Lean management, connecting company leadership to the ground truth of production. However, there is a lack of a common business-level language for visualising software value streams. The data in tool repositories is the equivalent ground truth of software delivery.
• Epiphany 1: Software productivity declines as software scales due to disconnects between the architecture and the value stream. Realignment of daily work around the value stream instead of the software architecture increases productivity.
• Epiphany 2: Disconnected software value streams are the bottlenecks to software productivity at scale. These disconnects are caused by the misapplication of the project management model.

• Two types of complexity in tool networks:
• Fundamental: Heterogeneity that improves flow of value by supporting needs of specialist stakeholders.
• Accidental: Heterogeneity that doesn’t improve the flow of business value.
• Reducing accidental complexity should be an ongoing effort, standardise where possible.

• The key differences between software development and manufacturing value streams:
• Variability: Manufacturing aims to minimise variability, software development has to embrace it.
• Repeatability: Manufacturing is about maximising throughput of the same widget, software is about maximising iteration and feedback loops that continually reshape the widget. The process needs to be optimised for flow, feedback and continual learning not just repeatability.
• Design frequency: Physical products are designed up front, where as in software the design happens inside the system.
• Creativity: Manufacturing increases automation and removes creativity from production. Software focuses on enabling creativity and collaboration at each step.

• Three layers of abstraction help connect the implementation details of the tool layer, to a more abstract business value-orientated view provided by value stream metrics.
• The Integration Model: The bottom layer is the tool network. The nodes are tools, and the links are cross-tool integrations.
• The Activity Model: The second later is the artefact network, nodes are artefacts (like tickets and defects) and they are connected by relationships between artefacts.
• The Product Model: The value stream network aligns the value streams to the business outcomes.
• The focus of the Flow Framework is to enable the Value Stream Network, which is what provides the business-level insights into software delivery.
• The connectivity index: The ratio of repositories in the tool network that have been connected to those that have not.
• Traceability Index: the measure of artefact connection breadth and depth relative to artefact type. Aiming for full visibility with a target of 100%.
• Alignment index: the ratio of artefact containers connected to a product value stream relative to all artefact containers across the tool network
• 5 key sources of waste identified by Dominica DeGrandis in her book Making Work Visible:
1. Too Much Work in Process: Overloading teams with too much work leads to inefficiency and reduced flow, as it becomes challenging to push work back onto the business.
2. Unknown Dependencies: Unplanned work often arises due to unrecognised dependencies between teams, components, and products.
3. Unplanned Work: Unanticipated work, which includes emergencies and rework, disrupts planned activities and flow, leading to delays and bottlenecks.
4. Conflicting Priorities: When different parts of the business have conflicting goals, it creates challenges in decision-making and prioritisation, affecting flow and delivery.
5. Neglected Work: Ignored or deferred tasks, such as technical debt or "zombie projects," result in waste and hinder efficient flow through the value stream network.

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Introduction

• Traditional organisations often recognise the need to transform, but they don’t know how.
• Reorganisations and transformations often focus on cost reduction, not business outcome.
• Agile is a good approach for software but it doesn’t mesh with traditional business frameworks (planning, budgeting, tracking) that treat software projects like everything else.
• The Flow Framework aligns technology investments to value streams that define the activities that happen to bring value to the market.
• The DevOps handbook introduced the three ways of DevOps: flow, feedback and continual learning. The flow framework scales them across your entire business.
• A Value Stream Network is way of visualising the flow of value across an organisation. It helps us understand and measure software investment and activities and connect them to business value.
• Don’t focus on transformation, focus on business results.

Part 1: The Flow Framework

• Most organisations have no notion of value streams or measurement of how business value is delivered.
• The software delivery efficiency of most companies is extremely poor when compared to digital startups or tech giants.
• There’s a huge difference between building physical and digital products. We must take inspiration from the history of the manufacturing industry, without blindly following in their footsteps.

Chapter 1: The Age of Software

• New waves of technology are disrupting existing businesses and the trend is accelerating.
• Each of the four major sectors identified by Zoltan Kenessey are undergoing rapid transformation, although at varying speeds.
1. Primary (resource extraction): Slow
2. Secondary (manufacturing): Fast
3. Tertiary (services - e.g. retail, banking, transport): Faster
4. Quaternary (knowledge based - e.g. technology, consulting, R&D): Fastest
• In many industries, software-based innovation is becoming the key factor for success, surpassing the importance of incremental product gains in other areas of the business.
• BMW expects 50% of its workforce to be building software.
• Geoffrey Moore in ‘Zone to Win’ introduced thee different types of disruption. The type of disruption can inform how you shape technology investments and value streams.There are three types of disruption that can inform technology investments and value streams:
1. Infrastructure Model: This type of disruption alters customer access to a product or offering without changing the sales process. For example, promoting products through social media.
2. Operating Model: This type of disruption utilises software to transform the interaction between consumers and the business. For instance, providing a top-notch mobile experience for airlines to maintain passenger bookings. It relies on using software to change the relationship of the consumer with the business.
3. Business Model: This type of disruption involves a significant change spurred by technology, such as the music industry transitioning from physical products to digital subscriptions.
• By identifying the type of disruption, organisations can make informed technology investments and align their value streams accordingly.
• Moore states established businesses are not capable of disrupting themselves.
• The other vector of disruption is the tech giants that have mastered software production.

• In any technology wave, there are two important periods: the Installation Period and the Deployment Period.
• The Installation Period involves deploying large amounts of financial capital, like venture capital, to disrupt the previous age.
• The Deployment Period is when companies that master the means of production start dominating the economy and infrastructure.Production capital, controlled by established companies, displaces startups and financial capital during the Deployment Period.
• Governments start imposing regulations as the new masters of production amass wealth and control around the Turning Point. This starting to happen with the tech giants we have today.

Three Epiphanies

1. Productivity declines and waste increases as software scales due to disconnects between the architecture and the value stream.
2. Disconnected software value streams are the bottleneck to software productivity at scale. These value stream disconnects are caused by the misapplication of the project management model.
3. Software value streams are not linear manufacturing processes but complex collaboration networks that need to be aligned to products.

• Research on the disconnect between architecture and value streams in software development shows as software systems grew, so did the distance between them.
• The traditional model of a linear manufacturing process for software value streams is inadequate. A better approach is a complex collaboration network that aligns to business objectives.
• The gap between business and technologists is widening, and a better approach is needed to bridge it.

Chapter 2: Project to Product

• Survival in the Age of Software is dependent on an organisation's ability to deliver software products and digital experiences.
• Digital transformation represents a significant opportunity, estimated at $18.5 trillion by 2020.
• Many organizations have initiated transformation initiatives, while others are realizing the increasing importance of software investment.
• In the Age of Software, digital technology has become the core of organisations and cannot be isolated to a separate department.
• Moving from project-oriented management to product-oriented management is crucial for connecting software delivery to the business and achieving success.

Agile Transformation Failure at Nokia

• Agile methods were applied to software development at Nokia, but there was an overemphasis on process adherence rather than outcome.
• Developers faced issues due to the Symbian OS architecture and long build/test/deploy cycles.
• The Agile transformation lacked clear, well-defined, and automated production metrics.
• Nokia's Agile transformation appeared successful in terms of activities, but it failed to deliver actual business results.
• Nokia's inability to transition from hardware-focused handsets to a software-centric mobile experience contributed to its downfall.
• Lack of continuous integration, delivery capability, outdated architecture, and disconnect between business and development teams were key issues.
• The narrow-minded, activity-focused approach to Agile was the root cause of Nokia's failed digital transformation.
• Lesson One: To avoid the pitfalls of local optimisation focus on the end-to-end value stream.
• The "end-to-end" concept in software value streams covers the whole process from business strategy to customer usage; identifying bottlenecks throughout is crucial before optimizing segments like feature design or deployment.
• Unlike BMW's Leipzig plant, which was built to visualize and adapt to bottlenecks, Nokia, despite its expertise in hardware, failed to apply these principles to software delivery, leading to a disconnect between business and IT and setting the stage for a failed digital transformation.

DevOps to the Rescue

• Blaming Agile or Scrum for Nokia's software transformation failure is flawed.
• Nokia's problems were not with Agile or Scrum, but with issues beyond the boundaries of Agile development teams.
• Investments made in areas other than the bottleneck are futile.
• Nokia's Agile transformation lacked clear metrics and failed to measure outcomes.
• Adopting DevOps practices like continuous delivery may have helped, but it wouldn't have altered Nokia's decline curve.
• Misalignment between business and development, tangled architecture, and lack of visibility in the value stream were key issues.
• Nokia lacked the infrastructure and visibility to see what was happening in their value stream.
• Had Nokia adopted the Three Ways of DevOps they would have at least started on the path to identifying the bottleneck.
• By focusing on flow and feedback from Dev to Ops, they might have seen indications of very long lead times for deployment.
• If they had elevated “continual learning” beyond engineering leaders it might have caused company leadership to start asking the right questions about org structure and software architecture.
• Treating DevOps as a technical practice rather than elevating it to the business would not have changed the outcome.
• Nokia's failure to apply Agile and DevOps at scale reflects broader industry challenges.
• Transformation failures stem from a misunderstanding of core DevOps and Agile principles, not technology or process deficiencies.
• The consistent failure of such transformations suggests a fundamental flaw in the decision-making and visibility within organisational structures.

A Disconnect between the Business and IT

• Don’t manage technology as a cost centre and focus on cost reduction. Instead involve business stakeholders in the transformation.
• Digital initiatives should not be pursued separately from technology changes, as that leads to misalignment between business goals and technology capabilities.
• Always prioritise business value over cost cutting
• Lesson Two: If you manage a transformation according to cost alone, you will reduce productivity.
• LargeBank's transformation, focused on cost reduction, missed key business outcomes.
• Treating IT as a cost center led to a disconnect, valuing cost-cutting over delivering value.
• Sophisticated metrics used in LargeBank's Agile and DevOps transformations were activity-focused, not results-oriented.
• Over-reliance on proxy metrics can mislead decision-making, overshadowing the need for metrics that truly reflect business outcomes.
• Jeff Bezos advocates resisting proxies, emphasising the importance of outcome-based metrics over process-based proxies.
• The problem in IT is the lack of agreed-upon, outcome-based metrics, leading to a reliance on insufficient proxies for success.

From Puzzles to Planes

• LargeBank's approach differs from BMW Group Leipzig's successful transformation in the automotive industry.
• Unlike project-focused management, BMW and Boeing adopt a product-centric view, focusing on value delivery.
• Boeing's 777 and 787 Dreamliner projects exemplify their adept handling of complex, high-stakes production with long-term value.
• Boeing’s strategy includes putting key software engineers directly into high-stakes testing scenarios to ensure accountability and success.
• Boeing's commitment to traceability stems from their long-term vision, planning for decades of production and maintenance.
• Boeing's decision to rewrite brake software for the 787 Dreamliner was economically driven, considering long-term maintenance and compliance.
• The success of the Dreamliner questions how well technology organisations understand product development compared to Boeing’s long-term, product-centric approach.
• Lesson three: Engineering and the business must be connected.
• Software success extends beyond minimising costs to include revenue, user satisfaction, and other key results.
• Boeing treats plane development as a profit centre, focusing on long-term profitability and the life cycle of its aircraft.
• Boeing's approach contrasts with many companies who view technology as a cost centre.
• Both Boeing and BMW exemplify a product-centric mindset, valuing long-term profitability over short-term cost reductions.

• Moore's "Zone to Win" outlines four investment zones: Productivity, Performance, Incubation, and Transformation.
• The Productivity Zone focuses on the bottom line with HR and marketing systems.
• The Performance Zone is top-line oriented, driving revenue.
• The Incubation Zone develops new products and businesses, focusing on monthly active users as a business objective.
• Businesses transition from the Incubation to Transformation Zone to scale and focus on profit.

• Organisations mistakenly use productivity metrics for measuring technology capabilities.

Project vs Products

• Project management has enabled notable achievements like the Hoover Dam, using the Gantt chart for labor efficiency and specialisation.
• Taylorism, though efficient, was criticised for treating workers as interchangeable resources, lacking autonomy and decentralisation.
• Henry Ford improved upon this by incorporating decentralised decision-making, leading to Fordism.
• Many IT organisations are stuck in the project-oriented world of Taylorism, causing communication gaps between business leaders and technologists.
• Software delivery is creative work, but managing technology as a repetitive process leads to futility in the digital market.
• The budgeting of projects is based on market and resource certainty, creating fixed goals and incentives for maximum budget requests.

• DevOps and Agile aim to address software delivery's inherent uncertainty, unlike traditional project metrics focused on cost.
• Project-oriented management can stifle market learning and iterative improvements due to its emphasis on defined time frames and budgets.
• Product-oriented management values the life cycle of products, allowing for continuous adaptation and value delivery.
• The focus on life cycle costs and profitability in product-oriented management contrasts with project-oriented management's cost centre approach.
• The project end fallacy overlooks software economics, notably technical debt, which can degrade software quality and add maintenance costs.
• Success metrics in project-oriented management create incentives for large projects and upfront resource allocation, contradicting lean and continuous learning principles.

• On risk: Traditional project-oriented management is not well-suited for software development because it requires up-front knowledge of all risks, which is difficult in the uncertain and changing world of software.
• Lean Startup and product-oriented approaches, such as minimum viable products (MVPs), are a better fit for software development because they allow for iterative learning and pivoting based on feedback.
• On teams: Project-oriented management, where resources are allocated to specific projects, is based on the Taylorist assumption that people are interchangeable and replaceable. This approach is unsuitable for complex knowledge work like software development, where building expertise and relationships takes time. Long-lived teams that stay together are more effective in this context. Bruce Tuckman coined the forming, storming, norming and performing team lifecycle model.
• Bringing work to people is more effective than bringing people to the work.

• On prioritisation: Traditional project management prioritises following the plan, even if it's expensive or inconvenient to change. This can lead to inflexible "waterfall" software development models. Product however sets priorities based on value hypotheses.
• Product-oriented management prioritises adapting to changing needs, based on continuous feedback and learning. This is a better fit for software development, where requirements are often unclear at the outset.
• On reporting progress: projects often look on track until they launch with disappointing results, then it’s clear they weren’t building the right thing.

Conclusion

• IT and digital transformation failures often stem from a business-IT disconnect.This disconnect manifests in:
• Project focus over product focus.
• Cost focus over profit focus.
• Timeline focus over delivering business value.
• Adopt a product-oriented approach to software delivery. Prioritise delivering business value over completing technical tasks. Embrace the Flow Framework for managing software delivery.
• The Flow Framework bridges the gap between business and IT. Focuses on measuring and managing the Value Stream Network (VSN). VSN: critical layer for successful software delivery at scale.
• Product-oriented management alone and the Flow Framework alone are insufficient. Success in the Age of Software requires:
• Managerial culture aligned with product mindset.
• Understanding of the rapidly changing market.
• Ability to bring software-based products to market and adapt.
• Defining how to measure business value in software delivery is crucial. The Flow Framework provides a way to achieve this.

Chapter 3: Introducing the Flow Framework

Introducing the Flow Framework

• Enterprise organisations struggle to effectively manage software delivery costs and lack visibility into this significant business expense.
• Enterprises need to shift from managing IT as projects or cost centers to adopting a product-oriented mentality.
• Technologists within enterprises are advocating for the adoption of DevOps and Agile practices to drive transformation.
• The principles of modern software-delivery approaches are not fully translating to the business.
• A new framework is required to apply Agile and Lean practices to the business and define outcome-oriented metrics.
• Businesses should adapt to a product-oriented mindset that supports the differences between physical widgets and software components.
• Activity-oriented proxy metrics should be replaced with business outcome-oriented metrics.
• The Flow Framework connects business and technology, enabling an organization-wide feedback loop to accelerate the flow of business value to customers and facilitate organizational learning in the evolving market of the Age of Software.

• Businesses struggle to manage software delivery with outdated methods, leading to high costs and missed opportunities.
• The Flow Framework bridges the gap between business and technology, enabling product-oriented software delivery.
• Product-oriented mindset: Shift from project-centric approach to viewing software as evolving products.
• Value Stream Networks (VSNs): Infrastructure providing automation and visibility for software delivery, similar to manufacturing production lines.
• Flow Items: Core elements tracked and optimized within the Flow Framework:
• Ideas: Capturing and prioritizing business needs for software solutions.
• Features: Units of functionality delivering business value.
• Releases: Bundles of features delivered to customers.
• Deploys: Moving features from development to production environments.
• Everybody in your organisation should be comfortable answering:
• Who is the customer?
• What value is the customer pulling?
• What are the value streams?
• Where is the bottleneck?
• The Flow Framework helps connect staff efforts and vision with organisational strategy.
• The benefits of the Flow Framework include:
1. Seeing the end-to-end flow of business value in real-time.
2. Instantly spotting bottlenecks and using them to prioritise investment.
3. Testing hypotheses with real-time data from every value stream.
4. Re-architecting your organisation to maximise flow.
• The Flow framework is about end to end measurement.
• Measurement of the work being done and the results it generates.
• The focus is on result-orientated businesses metrics (e.g. revenue) not proxy metrics for value (like lines of code, or number of deployments). Results > Activity.
• The framework is focused on end-to-end metric to identify bottlenecks wherever they lie in the value stream.
• The framework helps you track, manage and improve investments in automation and agility.
• 5 principles of ‘Lean Thinking’:
1. Precisely specify value by specific product
2. Identify the value stream for each product
3. Make value flow without interruptions
4. Let the customer pull value from the producer
5. Pursue perfection
6. as defined by Womack and Jones from their book of the same name.
• Identify the value stream for each product. Include every person, process, activity and tool related to delivering that software product:

Value Stream: the end to end set of activities performed to deliver value to a customer through a product or service

• The customer needs to be well defined too. Your customer can be internal (e.g. if your product is an API).

From Mapping to Architecture

• The ‘learning to see practice’ provides visual notation and a set of metrics for management of product flows and identification of waste and bottlenecks in production systems.
• The goal is to measure the end to end flow of business value → that can be correlated to business outcomes

Software flow: The activities involved in producing business value along a software value stream.

• You measure the flow of artefacts through the value stream network.

• Avoid measuring activity → instead track flow metrics that correlate to results.
• The flow framework creates a results driven feedback loop through…
• Value Stream Metrics that track each value stream within the organisation so you can correlate production metrics to business outcomes.
• The Value Stream network layer provides the infrastructure needed to measure the results delivered by each product.
• The Four Flow Items:
• Without an understanding of flow, you can’t identify the bottleneck that’s holding you back.
• There’s no measure of productivity for software development.
• Lean thinking starts with the value the customer pulls. To pull value, the customer must be able to see value and exchange something for it.
• For an internal product it might be adoption.
• For an external product it might be revenue or engagement time
• Flow Unit: A unit of business value pulled by a stakeholder through a product’s value stream.
• MECE flow units in a software value stream:
• Features
• Defects
• Risks (security, regulatory, compliance)
• Debts (technical debt)
• With full visibility, you could see every process and tool, and identify every contribution from designers, developers, managers and testers that were involved in creating, deploying and supporting a feature.
• Risks and debt are invisible to the user, they are pulled through the value stream by internal prioritisation (from the paper: Mining the Ground Truth of Enterprise Toolchains)

• Only debt that increases the flow through the value stream should be prioritised
• Prioritisation across the 4 flow units is a zero sum game.

• We’ve learnt a lot about software delivery, but we need to apply it to how businesses approach software projects.
• DevOps and Agile haven’t been adopted outside of technology teams.
• We need a new framework that bridges the gap between business and technology teams and enables the transition from project to product thinking.
• The framework must scale the ‘three ways of DevOps’ - flow, feedback, and continuous learning

Part 2: Value Stream Metrics

Chapter 4: Capturing Flow Metrics

• This chapter is all about measuring the first way of DevOps (end to end flow).
• Value streams and the four flow items provide us with an abstraction of business value flow in software delivery.
• We need to identify the key metrics that are most relevant to tracking the productivity of each value stream.

Understanding Flow Distribution

• We need to track the flow distribution of the 4 flow units.
• A new product requires a large proportion of features before launch.
• A legacy back-end service could have a value stream set to optimise for risk reduction and defect fixes (with minimal investment into feature flow)
• Flow Distribution: The proportion of each flow item within a value stream, adjusted depending on the needs of each stream to maximise business value.

1. Flow Metrics: velocity, efficiency, time, load
2. Business Results: value, cost, quality, happiness
3. Flow Distribution: features, defects, risks, debts
• Use flow distribution to tailor value streams to the results you need. Take into account product lifecycle.
• Teams on the value stream should know how to match flow distribution for business results.
• Flow distribution should be set consciously to achieve the company strategy.
• Legacy products approaching end of life should see no debt investment. They also shouldn’t be considered in any re-platforming.
• New products should be tuned to maximise features. But after launch the business needs to account for time spent on defects, risks and debts.
• Adding people to a team doesn’t create immediate returns. So any results needed in quick succession need to come from existing capacity.
• Flow distribution gives a zero-sum mechanism for deciding how one or more value streams should support business priorities.

• Flow distribution: allows the business to scale planning that product and engineering teams do on a daily basis.
• Thinking in flow distribution trade-offs elevates the understanding of the business trade-offs
• You can make visible the pain of delaying defect work and pushing out features to meet a business deadline.
• To see a balanced flow distribution, elements need to be of a similar size (e.g. the average feature might be 4x larger than the average defect). So you might choose to use stories not epics, so your size is more comparable.
• Flow Velocity: a measure of the number of each flow item being completed over time. Helps understand how much business value is being delivered to the customer.
• Velocity is across all flow items. If 10 features, and 5 risks are completed in a release, the flow velocity is 15.
• It differs from the traditional agile velocity measure because it doesn’t rely on estimation.
• You can use this to plan over time. E.g. feature velocity might be too slow to achieve what you want.
• Flow velocity is the empirical measure of productivity based on direct observation of value streams (it’s better than lines of code written, deploys per day, or time to go from commit to deploy)
• Flow items are tied to real business value (as product managers should be selecting the work that’s the most impactful)
• Flow items should have their value defined before they enter the stream.
• You don’t need to size flow items. It’s not about estimation its about making the flow of work and results visible. It all evens out when there’s a large number of them.
• Flow velocity metrics are more suited to track productivity and delivery trends within a value stream, rather than across value streams.
• Flow Time: to determine the speed of delivery we need to measure flow time.
• Lean manufacturing uses lead time and cycle time.
• Lead time: time through the entire process, is the best predictor of success.
• Cycle time: time it takes to complete a step within the process. Helps identify constraints. The longest is the bottleneck.
• Flow time was defined in Making Work Visible: Exposing time theft to optimise work and flow by Dominica DeGrandis.
• Flow time starts when a flow item is accepted into the value stream (a feature scheduled, or a bug reported). It is measured in wall clock time, until the item has been deployed to a customer.
• There are 4 flow states: New, Waiting, Active, Done.
• Lead time = New → Done
• MTTR(Mean time to repair) is the average time of all defects from New → Done.
• Lead time can be useless when there are many more requests or ideas sat in a backlog than can be actioned.
• Flow time = Active → Done
• Flow time is a measure of how long it takes to deliver a flow item, from the decision to take on the work to the value being delivered to the customer.
• Flow time is the most meaningful metric to tune to business results as it starts when the work has been accepted.
• You can have as many states as you like, just map them back to the 4 flow states.
• Flow load: represents concurrent work-in-progress (WIP) within a value stream. It is the number of flow items being actively worked on in a value stream.
• It’s an Indicator of potential issues impacting flow velocity and time.
• Excessive flow load signals inefficiency, linked to longer queue times and reduced output.
• The goal of tracking flow load is to fine-tune work items to optimise flow velocity and minimise flow time.
• Different value streams may tolerate different flow loads based on team and product maturity.
• Flow Efficiency: Measures the active work time on flow items against the total time elapsed in the value stream.
• Aims to reduce waste from items in waiting states, lowering efficiency.
• Flow efficiency drops with increased dependencies causing delays and bottlenecks.
• Identifying bottlenecks involves tracing back the causes of reduced flow efficiency.

Conclusion:

• Software products and value streams need to be as visible as in manufacturing lines.
• Challenges exist due to the amorphous nature of software products.
• Data collection and visualization tools are available, like the Leipzig plant’s digital dashboards.
• The issue lies in creating effective abstractions for visualization at the business level.
• DevOps teams have clarity on what to visualize, such as deployment metrics.
• Development teams use Agile methods like Scrum or kanban for visibility at the team and specialist levels.
• Flow metrics provide the needed visibility at the business level.
• Measuring flow metrics in real-time across all value streams is complex but necessary.
• Part III will outline how to create a Value Stream Network that supports flow and feedback.
• Defining the business results from tracking flow is essential before proceeding.

Chapter 5: Connecting to Business Results

• The second way (Feedback) is the process of connecting software production flow metrics to outcome-based business results in order to create a feedback loop. Allowing for continual learning at the organisation level that should help with planning.
• Business results should be tracked continually for each product-oriented value stream (most organisations track ‘by project’).
• You need to track each of these measures for each value stream:
• Value: measures benefit to the business produced (e.g. revenue, MRR, 7DAU).
• Use revenue if you can or a metric that that’s a good proxy
• You can guess revenue for pre-revenue streams, or use pipeline growth metrics
• If internal application, use a value outcome metric. Dependencies are later mapped in the value stream network, so it will be clear what revenue-getting streams they support.
• Cost: of the value stream to the business (cost of all staff, operations and infrastructure)
• It’s important that people are allocated to a single value steam.
• Shared people should be the exception not the norm.
• Shared resources should have a sensible allocation to the value stream.
• Quality: of the product produced as from the customer perspective (escaped defects, NPS, retention)
• Value streams focus on the customer. So the DORA metrics are less relevant here.
• Tracking separately for each value stream makes quality tradeoffs visible.
• Happiness: engagement of the staff on the stream (eNPS, employee engagement).
• Project-orientated management gets in the way of autonomy, mastery and purpose… product-orientated stability promote them.
• Lack of staff happiness can be an important indicator of a problem with production.
• Measuring value and cost allows us to measure the life cycle profit for each value stream.

• As flow metrics are correlated to business results, if high feature flow isn’t translating into business value, there’s either a problem in marketing/sales or a problem with product/market fit.
• Conclusion:
• The Flow Framework aims to track business results for each value stream and correlate investments with outcome-based metrics.
• Transitioning from a project-oriented model to a product-oriented model is necessary to align value streams with products and IT portfolios.
• Canonical metrics for each value stream include measures of value, cost, quality, and happiness.
• Creating a value stream dashboard with common Value Stream Metrics can provide transparency and inform decision-making.

Chapter 6: Tracking Disruptions

• Cars have been evolving with increasing lines of code, from one million for basic drive features to one billion with the advent of autonomous drive.
• Trend of software-related recalls due to the Operating Model disruption of the car industry and pressure to bring new features to market.
• The Flow Framework hypothesis suggests that too much flow distribution allocated to features has resulted in excessive technical debt, impacting quality trade-off and recall rates.
• Risk problems at Equifax highlight the need for prioritizing risk and debt reduction in value streams.
• Debts and the downfall of Nokia demonstrate the impact of technical debt on product quality, feature velocity, and overall success.
• Microsoft's product-oriented management approach and prioritization of internet-centric features secured its future, with deliberate trade-offs made for quality and technical debt.
• Gates' Trustworthy Computing initiative at Microsoft focused on reducing risk and prioritizing security before it became a common concern.
• BMW's and Microsoft’s thrive by closely linking business strategy with production methods.
• Nokia suffered from a lack of focus on technical debt, leading to significant problems.
• The Flow Framework can be the decision-making framework that enables leadership to recognise the right time for strategic moves like re-platforming, which can be pivotal for a company's survival.
• Many successful tech leaders come from software engineering backgrounds, are they more likely to balance technical and business considerations effectively?
• There needs to be a common language that bridges the gap between technologists and business stakeholders to enable better decision-making.
• The Flow Framework provides a comprehensive approach to managing organizations that can prevent such issues.

Part 2: Conclusion:

• Defining flow items: Each flow item can be measured by observing the tool network. With the flow items, we can measure the flow of business value through the tool network and correlate that to business results.
• Setting the flow distribution: This is the most consequential aspect of the Flow Framework, as all other metrics rely on it. We need to track the target flow distribution for each product’s value stream to determine what type of business value is being delivered.
• Measuring flow velocity: We need to be able to measure the magnitude of each flow item being delivered to the customer over time. This is the flow velocity, and it is adapted from the concept of velocity in Agile software development.
• Tracking flow time: Flow distribution and flow velocity provide an empirical measure of how much of which kind of work was done over a period of time, but they do not indicate how quickly that work cycled through the system. Flow time defines the speed at which we are delivering business value to the market.
• Measuring flow load: In order to optimise flow in a value stream, we need to avoid over-utilisng the value stream by having too much work in progress (WIP). The flow load metric allows us to track this at the value stream level (e.g., by indicating how many features are being worked on in parallel).
• Tracking flow efficiency: Within each product’s value stream, flow items are either being worked on or waiting for work to be done or for dependencies to be addressed. Flow efficiency measures the ratio of productive to waiting time, enabling the tuning of our value streams for productivity.
• Connecting to business results: Value, cost, quality, and happiness are the four buckets of business results that need to be tracked as part of the Flow Framework in order to correlate software investment with business performance, and to provide a common set of results-oriented metrics to connect the business with IT.

Part 3: Value Stream Networks

Chapter 7: The Ground Truth of Enterprise Tool Networks

• The Gemba walk is a fundamental part of Lean management, connecting company leadership to the ground truth of production. However, there is a lack of a common business-level language for visualising software value streams. The data in tool repositories is the equivalent ground truth of software delivery.
• Epiphany 1: Software productivity declines as software scales due to disconnects between the architecture and the value stream. Realignment of daily work around the value stream instead of the software architecture increases productivity.
• Epiphany 2: Disconnected software value streams are the bottlenecks to software productivity at scale. These disconnects are caused by the misapplication of the project management model.

Chapter 8: Specialised Tools and the Value Stream

• Disconnected tools cause fragmentation of the value stream and impact developer productivity.
• Software delivery is too complex for one tool, one team or even one organisation. Make sure the tools support the flow.
• As software development has scaled, practitioners specialise their roles and tools.
• Two types of complexity in tool networks:
• Fundamental: Heterogeneity that improves flow of value by supporting needs of specialist stakeholders.
• Accidental: Heterogeneity that doesn’t improve the flow of business value.
• Reducing accidental complexity should be an ongoing effort, standardise where possible.
• Instances for fundamental complexity:
• Stakeholder specialisation: stakeholders require different tools in order to be effective at their discipline (e.g. developers for committing code, support staff for tracking tickets)
• Scale specialisation: some tools have limits to organisational size and scale
• Platform specialisation: vendors provide a add ons to their platforms to increase lock-in
• Zone specialisation: Early stage incubation products can have more light weight process tools
• Legacy: cost disruption to moving away from legacy systems can be prohibitive
• Supplier diversity: connecting across organisational boundaries can fix tool choice (e.g. open source use open source tools)
• Dimensions of scale that effect tooling complexity:

• Companies can benefit from a highly simplified product offering that can limit complexity.
• There’s often thrashing in the parts of the tool network that store flow items and that defines how work flows between people and teams.
• Often hand-offs are happening manually. Detail gets lost. Work should flow to completion. Results in delays and rework.
• Connecting the value stream network can help.
• You need to get access to the tool-repository data.
• Crete a Value Stream Integration diagram with each of the tools used. Containing the repositories, artefact types stored and the data of how those are artefacts are related in terms of flow.

• We’re seeing a wave of tool specialisation not consolidation.
• Many legacy tools in the space are hard to deprecate and they go on being used for years.
• There are two exceptions to the spread of tooling: Startups who don’t have the scale, and use fewer tools. And tech giants, like Google who use in-house tools (that require large investments).

Chapter 9: Value Stream Management

• We should be able to see the flow value as clearly as car manufacturers and identify bottlenecks in the system.
• The concepts of flow and transparency translate directly from manufacturing to software.
• In software though the product development and manufacturing processes are one. This is why you can’t reduce the software delivery process to a manufacturing line!
• In software you’re able to re-route around bottlenecks more easily.
• Epiphany 3: Software value streams are not linear manufacturing processes but complex collaboration networks that need to be aligned to products.
• Waterfall and Agile have limitations when applied to large-scale software delivery.
• Waterfall's linear approach failed in practice due to its disconnect from various stakeholders.
• Agile and DevOps improved aspects of delivery but may oversimplify by focusing too much on linear processes.
• Software development is likened to manufacturing with its batch flow, automation, and repeatability as key success factors.
• Software processes are more network-like than linear, resembling an airline network rather than a production line.
• Understanding IT organisation workflows requires a network view, similar to airline traffic visualisations. To optimise software delivery, it's important to first understand and then manage the underlying network of project, product, and team interactions.
• Many lean concepts still apply (small batch sizes, minimising work in practice)
• The key differences between software development and manufacturing value streams:
• Variability: Manufacturing aims to minimise variability, software development has to embrace it.
• Repeatability: Manufacturing is about maximising throughput of the same widget, software is about maximising iteration and feedback loops that continually reshape the widget. The process needs to be optimised for flow, feedback and continual learning not just repeatability.
• Design frequency: Physical products are designed up front, where as in software the design happens inside the system.
• Creativity: Manufacturing increases automation and removes creativity from production. Software focuses on enabling creativity and collaboration at each step.
⚫
Manufacturing is about producing the same thing many times → Software development is actually the opposite. No code should be written twice.

Visualising the Value Stream Network

• Three layers of abstraction help connect the implementation details of the tool layer, to a more abstract business value-orientated view provided by value stream metrics.

• The Integration Model: The bottom layer is the tool network. The nodes are tools, and the links are cross-tool integrations.
• The Activity Model: The second later is the artefact network, nodes are artefacts (like tickets and defects) and they are connected by relationships between artefacts.
• The Product Model: The value stream network aligns the value streams to the business outcomes.
• The focus of the Flow Framework is to enable the Value Stream Network, which is what provides the business-level insights into software delivery.

Connecting the tool network

• The tool network includes hosted and on-premise tools that support value stream artefacts.
• Flow and feedback in value streams are enabled by connecting these tools.
• The connectivity index measures the integration ratio of tools and artefact containers within the network.
• The connectivity index: The ratio of repositories in the tool network that have been connected to those that have not.
• A low connectivity index indicates incomplete integration and potential information bottlenecks.
• Full integration is crucial for accurate flow metrics, especially for customer-support originated flow items.
• Disconnections in tool networks hinder end-to-end information flow across value streams.
• The connectivity index is vital for ensuring information can flow seamlessly end-to-end.

Connecting the Tool network with the integration model

• Integration of multiple tools in the tool network is crucial to support workflow and feedback within a value stream.
• The Integration Model maps cross-tool work items and artifacts, aligning them with the four main flow items for effective reporting and management.
• Over 200 workflow states on a single artifact can complicate visibility; the Integration Model abstracts this complexity.
• A high connectivity index signifies better integration and data flow across tools, which is essential for reliable flow metrics.
• The Integration Model acts as an abstraction layer, allowing disparate tools to connect and communicate effectively.
• Proper integration can lead to transformative effects on an organization, such as improving employee engagement by reducing friction.
• The model provides a modular approach to IT tool networks, enabling easier onboarding of new tools and reorganisation.

Producing the artefact network

• The Integration Model defines how business value flows through the Value Stream Network, akin to a hierarchy of artefacts.
• It creates an artefact network, ensuring visibility of work across multiple value streams.
• Ensuring every artefact in the network is connected maximises end-to-end visibility; disconnected artefacts from unaligned methodologies remain visible but are not mapped to flow metrics.
• Traceability Index: the measure of artefact connection breadth and depth relative to artefact type. Aiming for full visibility with a target of 100%.
• The index reflects traceability automation within the Value Stream Network, essential for governance and compliance.
• Automated traceability of artefacts from initial request to feature release simplifies reporting and auditing.
• Artefacts pass through various phases as defined by the Integration Model, which are mapped to specific activities in the value stream.
• The phases of the artefacts are aligned with the organisation’s software delivery and business processes, allowing for tracking through their lifecycle.

Aligning the value stream network with the product model

• The Value Stream Network is the topmost layer for automated flow and feedback between tools and specialists, removing manual work and providing transparency.
• The Product Model resolves the disconnect between software architecture and business-level product offerings by mapping tool networks to product value streams.
• There is often a misalignment between Agile tools structure and the modular design of programming languages or legacy software evolution.
• The lack of alignment between organisational and team structures, software architecture, and value stream architecture is identified as a barrier to effective software delivery.
• Feature teams and modularity in programming show promise for aligning the artefact containment structure with business value streams.
• The Product Model maps existing structures in the tool network to product-oriented value streams.
• The alignment index is part of the Flow Framework, measuring how completely the Product Model is mapped across the organisation, indicating visibility and clear mapping to business results.
• This index assesses the portion of work connected to value streams, aiming for high visibility and mapping accuracy.
• Alignment index: the ratio of artefact containers connected to a product value stream relative to all artefact containers across the tool network.

Making value streams visible

• 5 key sources of waste identified by Dominica DeGrandis in her book Making Work Visible:
1. Too Much Work in Process: Overloading teams with too much work leads to inefficiency and reduced flow, as it becomes challenging to push work back onto the business.
2. Unknown Dependencies: Unplanned work often arises due to unrecognized dependencies between teams, components, and products.
3. Unplanned Work: Unanticipated work, which includes emergencies and rework, disrupts planned activities and flow, leading to delays and bottlenecks.
4. Conflicting Priorities: When different parts of the business have conflicting goals, it creates challenges in decision-making and prioritization, affecting flow and delivery.
5. Neglected Work: Ignored or deferred tasks, such as technical debt or "zombie projects," result in waste and hinder efficient flow through the value stream network.

Conclusion:

• The final components of the Flow Framework have been defined to create an end-to-end flow for software delivery, leveraging a Value Stream Network and integrating tool networks.
• The Integration Model abstracts over the tool network to facilitate automated information flow across roles and teams.
• The Activity Model maps interactions to flow items and value stream stages, while the Product Model aligns activities with software products for measuring and correlating flow metrics to business results.
• The Value Stream Network offers a business-centric and customer-centric viewpoint for delivering software products.
• The overarching goal of the Value Stream Network is to foster innovation in software delivery at scale, transitioning from project-based to product-based organisation, and enhancing business visibility and software delivery optimisation.