Review

This book is consistently referenced by people in the technology industry, but few have read it. It was written in 1975 - how we build software products has changed a lot since then. Some of the observations in this book are timeless, but equally if you’re a Product Manger there are many chapters and pages you should skip (unless you’re keen on understanding the memory management challenges of yesteryear).

I think a lot about what books and material to read, I’m aiming to read more timeless texts and reduce my recency bias. That said - reading books about software development written 50 years ago is a bit of an art in and of itself.

You Might Also Like…

Shape Up

Shape Up is an incredibly refreshing read. It’s opinionated, new and different. Ryan does a great job of describing a delivery system, that addresses some of the most common challenges in building and shipping software. Adopting the methodology lock-stock-and-barrel will be too much for most teams, but it has you wondering what could I borrow? Great read.

andrewclark.co.uk

Book Summary: Accelerate by Nicole Forsgren, Jez Humble, and Gene Kim

Get a comprehensive review of the best-selling book "Accelerate" by Nicole Forsgren, Jez Humble, and Gene Kim. Discover the key insights and takeaways from this must-read for tech leaders and professionals

andrewclark.co.uk

Book Review and Summary: Inmates Running the Asylum by Alan Cooper

Get a comprehensive review and summary of the best-selling book "Inmates Running the Asylum" by Alan Cooper, the software development expert and author. Learn about the key principles and practices for creating user-friendly software and avoiding common pitfalls in the development process.

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

• Many large software projects have been swallowed. Some emerge with running systems, few meet goals, schedules and budgets.
• No one thing seems to cause the difficulty. The accumulation of simultaneous and interacting factors brings slower and slower motion.
• We hear and believe tales of a couple of programmers in a garage creating software faster than larger teams in offices.
• Programming is fun because:
• the sheer joy of making things
• the pleasure of making things that useful to other people
• fascination of fashioning complex puzzle-like objects
• the joy of always learning
• The woes of the craft:
• One must perform perfectly (computers are strict)
• One rarely controls the circumstance on ones work, or its goal
• Dependence upon others and their programs
• Designing grand concepts is fun, fixing bugs is just work
• Closing out bugs is not linear, its quadratic approach that seems to take forever
• Most software projects go awry due to lack of calendar time (vs all other causes combined)
• We are bad at estimating
• Our estimating techniques confuse effort with progress, hiding the assumption that men and months are interchangeable
• Progress is poorly monitored
• When we detect a delay, we add more people (which can make things worse not better)
• Most programmers are optimists (probably because learning to program is so frustrating only the optimists remain). This time it will surely run.
• The probability of a large programming project going well, with lots of tasks that depend on each other needing to go well is very small
• The person-month as a unit for measuring the size of a job is dangerous and deceptive. It implies that men and months are interchangeable.
• This would only be true if tasks can be perfectly divided and there be no communication overhead
• This is true of reaping wheat but not systems programming
• When a task can't be partitioned, more people don't help (it takes 9 months to make a baby)
• When a task can be partitioned, but with communication overhead, you need to add the overhead to the work.
• Theres both training and intercommunication. Intercommunication is tricky. The number of nodes increases exponentially (the power of network graphs in reverse). Dumbar's number style effect.
• Since software construction is inherently a systems effort, an exercise in complex interrelationships communication effort is great, and dominates the effect of adding more people to the project.
• We expect there to be no bugs.
• Failure to allow enough time for testing is disastrous. Since the delay comes at the end of the schedule, no one is aware of schedule trouble until the delivery date. Cost per day is also normally at its highest at this point
• Urgency may govern the scheduled completion of the task, but it cannot govern the completion. As with cooking an omelette. If its not ready, you have 2 choices. Eat it raw or wait.
• Adding manpower to a late software project can make it later (training and repartitioning of the system)
• People often prefer small teams of talented people. The number of people being coordinated has a communication cost (and correcting the mistakes made in communication - bugs)
• Variance of productivity between the best and worst programmers is huge. 10:1 between the best and the worst . No correlation between experience and performance.
• One therefore wants to build the system with as few people as possible?
• If a 200 person project has 25 managers who are the most competent and experienced programmers, fire the 175 troops and put the managers back to programming
• The small sharp team concept is just too slow for big projects.
• The Team Size Dilemma: For efficiency and conceptual integrity, one prefers a few good minds doing design and construction. .Yet for large systems, one wants a way to bring considerable man power to bear, so that the product can make a timely appearance.
• Mills's Proposal → Each segment of a large job can be tackled by a small surgical team
• Instead of everyone cutting away at the problem, one does the cutting.
• Everyone else gives him the support needed
• Few minds involved in design and construction, many minds brought to bare…
• Surgeon, copilot, administrator, editor, 2 secretaries, program clerk, toolsmith, tester, language lawyer
• To build things that take 5k person years you need multiple surgeons who have to spend all of their time talking to each other about the system architecture.
• Conceptual integrity is the most important consideration in system design. Cathedral design often changes in construction. It's hard to get generations of people to stick to the original design. Reims managed this.
• Conceptual integrity in turn dictates that the design must proceed from one mind, or from a very small number of agreeing resonant minds.
• BUT Schedule pressures dictate though that we need many builders. So we need either:
• Careful division of labor between architecture and implementation
• New way of structuring programming implementation teams discussed in the previous chapter
• Implementers raise 3 objections to architects doing the design:
• Specifications would be too rich in function and will not reflect practical cost considerations
• Architects will get all the creative fun and shut out the inventiveness of the implementers
• Many implementers will have to sit idly by while the specifications come through the narrow funnel that is the architecture team
• The Second-System Effect: There's a tendency of architects to over design their second large system. Using all the frills and ideas that were cautiously sidetracked on the first one. Be wary of the special temptations of the second system
• Written Specifications are necessary tool - although not sufficient. It describes and prescribes every detail of what the user sees. It is the chief product of the architect. Feedback cycle - from users and implementers allows you to iterate on the manual. Implementers need there to be strict versioning. The manual must describe what the user sees, and refrain from talking about what they don't see. That is the implementers business. His design freedom must be unconstrained. The style must be precise, full and accurately detailed.
• Formal Definitions. Language is not naturally precise.Need a formal definition and a prose definition. A formal definition is an implementation and an implementation is a formal definition (ask the machine)
• The purpose of the organisation is to reduce the amount of communication & coordination necessary. Division of labour and specialisation of function. Tree like structure reflects the need for diminishing need for detailed communication. A tree is a structure of authority and responsibility. The communication structure is should not be limited to single lines. The communication structure should be a network not a tree.
• Thinkers are rare, doers are rarer and thinker-doers are rarest.
• Data for building isolated small programs are not applicable to programming systems products.
• The bigger the product the harder it is to build and test. You can't extrapolate 100m sprints to a 3 minute mile
• Typically programs of half the size take a quarter of the time
• Effort goes up as a power of size even when no communication is involved except that of a man with his memories.
• Portman's Data found programming teams take 2x as long as they think. The miss was because the teams spent only 50% of the working week programming, the rest was debugging. Machine downtime. Other jobs meetings etc. We overestimate the amount of time we get to spend on programming. Aron’s data …the more interactions the fewer lines of code produced.
• The reasons for writing decisions down:
1. Gaps in your own logic appear
2. Communicate to others
3. Give you a database and checklist. You can review them to see where you are.
• Plan to Throw One Away. With most complex systems, the first system built is rarely usable (performance flaws).You'll have to make some major changes and rebuild. The only question is whether to: plan in advance to build a throwaway OR promise to give a throwaway product to paying customers.
• The only constant is change itself. Once you realise you need to build and throw away a pilot. It's clear you should embrace change. Accept change as a way of life. The threshold for changes must be raised overtime, otherwise no product ever emerges.
• Plan the system for change… Modularisation etc
‣
Plan the organisation for change…
• Bugs are 40% of the cost of building a system. The more users you have, the more bugs they’ll find. There’s a delay to bugs being found after release. Every time that you fix a bug, theres a 20-50% chance you introduce one=
• Repairs increase entropy and disorder of the system. Less and less effort is spent fixing original design flaws - more on fixing the fixes. Sooner or later, fixing ceases to gain any ground. Each forward step is matched by a backward one.
• Systems program building is an entropy-decreasing process, hence inherently metastable.. Maintenance is entropy-increasing process = which leads to unfixable obsolescence.
• The crucial task is to get the product defined. Many failures come from areas of the product that were not quite defined.

• The Flow Chart Curse. Oversold piece of documentation. Keep to one page if you have to use them. Don't do every detail - its a really space hogging way to describe something
• Using rapid prototyping as part of planned iteration in establishing requirements
• Grow software organically, adding more and more function to systems as they are run, used and tested
• The Essence = difficulties inherent in the nature of software
• Specification, design and testing
• Complexity: Software is complex - there is no repetition (that's all captured in the software itself). Often complexity is non-linear with size.
• Conformity: Much complexity comes form conformity to other interfaces
• Changeability: Pressures of change. Recalls are uncommon in hardware, but ubiquitous in software. All successful software gets changed. Once you get users, you get new requirements. Successful software also survives beyond the life of hardware or platforms, so it must adapt to survive
• Invisibility: Software is not embedded in space, so it's hard to visualise.

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Chapter 1: The Tar Pit

• Analogy is dinosaurs struggling in a Tar Pit (like quick sand)
• Many large software projects have been swallowed. Some emerge with running systems, few meet goals, schedules and budgets
• No one thing seems to cause the difficulty. The accumulation of simultaneous and interacting factors brings slower and slower motion.
• We hear and believe tales of a couple of programmers in a garage creating software faster than larger teams in offices.
• A program is normally morphed into one of these two:
• Break it into different interfaces with agreed syntax and test the combinations
• Make a program so it can be run, tested and extended by anybody
• Programming is fun because:
• the sheer joy of making things
• the pleasure of making things that useful to other people
• fascination of fashioning complex puzzle-like objects
• the joy of always learning
• The woes of the craft:
• One must perform perfectly (computers are strict)
• One rarely controls the circumstance on ones work, or its goal
• Dependence upon others and their programs
• Designing grand concepts is fun, fixing bugs is just work
• Closing out bugs is not linear, its quadratic approach that seems to take forever
• Products can be obsolete before completion

Chapter 2: The Mythical Man-Month

• Most software projects go awry due to lack of calendar time than all other causes combined
• This is true because:
• We are bad at estimating
• Our estimating techniques confuse effort with progress, hiding the assumption that men and months are interchangeable
• Progress is poorly monitored
• When we detect a delay, we add more people (which can make things worse not better)

Chapter 3: The Surgical Team

• People often prefer small teams of talented people

The Problem:

• Variance of productivity between the best and worst programmers is huge
• 10:1 between the best and the worst
• There was no correlation between experience and performance
• The number of people being coordinated has a communication cost (and correcting the mistakes made in communication - bugs)
• One therefore wants to build the system with as few people as possible?
• If a 200 person project has 25 managers who are the most competent and experienced programmers, fire the 175 troops and put the managers back to programming
• The small sharp team concept is just too slow for big projects.
• There is a dilemma.
• For efficiency and conceptual integrity, one prefers a few good minds doing design and construction.
• Yet for large systems, one wants a way to bring considerable man power to bear, so that the product can make a timely appearance.

Mills's Proposal:

• Each segment of a large job can be tackled by a small surgical team
• Instead of everyone cutting away at the problem, one does the cutting.
• Everyone else gives him the support needed
• Few minds involved in design and construction, many minds brought to bare
• Surgeon, copilot, administrator, editor, 2 secretaries, program clerk, toolsmith, tester, language lawyer

Scaling up

• The problem though, is how to build things that take 5000 person years
• A 10 person team can be effective no matter how they're organised (if they're doing everything)
• The surgical team concept scales, because the surgeon is doing the design. So you only need to get the surgeons to talk to each other. That's how you put 200 people on a problem and face the problem of only coordinating 20 minds (of the surgeons)
• However, its still tricky to coordinate 20 people.
• You need a system architect to ensure the system will function
• The architect must confine himself to architecture.

Chapter 4: Aristocracy, Democracy, and System Design

Chapter 5: The Second-System Effect

If you separate architecture from building responsibility. What stops the architect from enthusiastically inventing too much? Well - communication between the two.

Chapter 6: Passing the Word

• Assuming that you have great architects.
• How can you make sure that everyone hears, understands and implements their decisions

Chapter 7: Why did the Tower of Babel Fail? (Project Workbook etc)

Chapter 8. Calling the shot

• The coding is only one sixth of the problem
• Planning time, coding, component test, system test.
• Data for building isolated small programs are not applicable to programming systems products.
• The bigger the product the harder it is to build and test
• You can't extrapolate 100m sprints to a 3 minute mile
• Typically programs of half the size take a quarter of the time
• Effort goes up as a power of size even when no communication is involved except that of a man with his memories.

Chapter 9: Ten Pounds in a five-pound stack

• Not really relevant. Talking about the cost of storage.

Chapter 10: The documentary hypothesis

• Examines the documents needed on different projects to see if there is a theme.
• What: Objectives + Product Specifications
• When: Schedule
• How much: Budget
• Where: space allocation
• Who: org chart
• The reasons for writing decisions down:
1. Gaps in your own logic appear
2. Communicate to others
3. Give you a database and checklist. You can review them to see where you are.

Chapter 11: Building tips, planning tips

Chapter 12: Sharp tools

• The principal problem is communication. Individual tools hamper rather than aid communication.
• Technology changes, so tool lifetime is short.
• More efficient to have common development and maintenance tools
• The manager of a project needs to set aside time for building common tools

Chapter 13: The whole and the parts

• How do you build a program that works?
• How do you test a program?
• How do you test a program as part of a larger system?

• The crucial task is to get the product defined. Many failures come from areas of the product that were not quite defined.
• Function definitions. Specifications. Exorcism of frills of function all reduce the number of bugs

Testing the spec

• Before code exists you can test a spec. Developers won't tell you they don't understand it so you can get a 3rd party involved
‣
Top-down design (with step refinement)

Structured programming: didn't understand this.

Chapter 16 - No silver bullet - essence and accident in software engineering

• Exploiting the mass market to avoid constructing what can be bought
• Using rapid prototyping as part of planned iteration in establishing requirements
• Growing software organicallying, adding more and more function to systems as they are run, used and tested
• Identifying and developing conceptual designers of the rising generation