Why Greatness Can't be Planned

Why Greatness Can't be Planned

Author

Kenneth O. Stanley and Joel Lehman

Year
2015
image

Review

The importance of objectives is often stressed in Product Management circles, but often with mixed results. This book is a reminder of the limitations of objectives, and the importance of allowing for more exploratory innovation.

image

You Might Also Like:

Impact First

Most product managers believe they're focused on impact, but Matt demonstrates there's actually a spectrum, and you're probably not obsessing enough over your team's true impact. This book identifies common pitfalls many organisations fall into and provides tangible questions to help extract yourself when you're stuck. A good read for Product aficionados.

Impact First
Book Review and Summary: Measure What Matters by John Doerr

Get a comprehensive review and summary of the best-selling book "Measure What Matters" by John Doerr, the business leader and venture capitalist. Learn about the key principles and practices for setting and achieving measurable goals using the OKR framework.

Book Review and Summary: Measure What Matters by John Doerr
Radical Focus

The second edition of Radical Focus is one of the better books on implementing OKRs. OKRs are really powerful in the right context. It’s refreshing for a proponent of OKRs to come out and say they’re not useful everywhere. Christina doesn’t recommend their use in operations teams and proposes subtle adaptations of them for exploratory or long-lead-time work. There are many good OKR explainer books, but I feel this one does a great job of describing some of the rough edges and challenges.

Radical Focus

Key Takeaways

The 20% that gave me 80% of the value.

Modern life is saturated with objectives. From school tests to fitness targets, almost everything is framed as a measurable pursuit. The standard pattern is: set a clear objective, work hard, and measure progress along a gradient of improvement. This works well for modest goals: incremental efficiency and minor product upgrades) therefore we generalise and assume it’s the right way to pursue anything.

The book argues that this assumption silently breaks down for ambitious aims: deep innovation, major discoveries, or genuine happiness. In large, unknown “search spaces” of possibilities, we don’t know the stepping stones in advance, and they rarely resemble the final goal. Vacuum tubes didn’t look like computers, engines didn’t look like airplanes, and radar didn’t look like microwave ovens. Objectives become deceptive: they act as a false compass, pulling attention toward moves that look closer to the goal while ignoring strange, essential stepping stones that don’t resemble it at all.

Achievement is better seen as discovery inside a vast landscape of possibilities. You navigate this landscape by stepping from one idea or experience to the next, constrained by where you already are. The most important stepping stones often look irrelevant or “off‑path” at the time, which means rigid goals and metrics can actively block access to them. Optimising test scores can undermine real learning.

In careers and life paths, the same pattern appears. People often begin with one clear objective and end up thriving somewhere unexpected: law to writing, music to acting, odd jobs to entrepreneurship. The common factor in success is not perfect planning, but a willingness to treat initial goals as provisional and pivot when a more promising stepping stone appears. Unplanned experiences (volunteering, side projects, new communities, strange hobbies) become critical because they expose you to stepping stones you couldn’t have designed into a plan.

Consider unstructured play. Activities pursued “just because they’re fun” often build skills, networks, and perspectives that later support surprising opportunities. Dismissing such pursuits as pointless misses the fact that we don’t know which stepping stones will matter. It is safer for society if some people commit deeply to paths most of us would ignore, because that’s how unusual discoveries enter the collective pool.

Picbreeder, the online “picture breeding” system, makes this visible. Users evolve images by repeatedly picking interesting variants. Those who fixate on a specific target image struggle; those who follow interesting shapes and change their aims as they go produce the most striking results. The system as a whole discovers rich images not because anyone aims directly at them, but because many people chase what looks interesting from their current positions.

This intuition is formalised in novelty search, an algorithm that ignores explicit objectives and rewards only behavioural novelty. In a hallway task, a robot driven by novelty explores many ways of crashing into walls before it “realises” that to keep being novel, it must avoid walls and eventually go through the door. Bad behaviour comes before good behaviour; complexity emerges because simple, repetitive behaviours are exhausted. In maze experiments, novelty search consistently found solutions while objective‑driven search got stuck in dead‑end cul‑de‑sacs that appeared to be progress. Not trying to reach the goal outperformed trying, because the goal metric kept guiding the agent into dead ends.

Novelty search is not a magic bullet. In huge or endless spaces it can wander forever and miss specific targets. Theory backs this limitation: No Free Lunch results show there is no universally best search algorithm. For distant, deceptive objectives, neither pure objectives nor pure novelty nor hybrids can guarantee success. The realistic conclusion is uncomfortable but important: no method can reliably reach arbitrary ambitious objectives.

Yet search is far from futile. While we can’t guarantee finding a particular treasure, we can reliably find some treasure. Evolution, human innovation, Picbreeder, and novelty search all behave as “treasure hunters”: they accumulate stepping stones without knowing where they lead and occasionally stumble into remarkable outcomes. The right stance is not “how do I get that exact thing?” but “how do I expose myself to many interesting stepping stones and keep moving?”

This has systemic implications. In science funding, expert panels rate proposals and fund those with the highest consensus scores. That rewards compromise projects everyone finds “good enough” and filters out ideas that sharply divide opinion, the very ideas most likely to sit on the frontier between the known and unknown. Similar objective thinking drives grand programmes to “cure X” or “win Y technology race,” and policies that favour projects with obvious near‑term impact aligned with national or commercial goals. But history shows that major breakthroughs frequently emerge from work that looked useless, obscure, or misaligned at the time: pure mathematics, odd biological studies, or curiosity‑driven experiments.

Importance, consensus, and predicted impact therefore act as deceptive objective functions in the scientific search space. They prune away weird, low‑status stepping stones that may later prove essential. A healthier funding ecology would reserve some resources for projects chosen for interestingness, not impact forecasts; reward disagreement among reviewers as a sign of novelty; and back people with a track record of unexpected discoveries without forcing them to promise realistic outcomes in advance.

Truly innovative ventures arise when someone notices that a surprising leap has just become one step away (e.g. battery technology makes an electric sports car suddenly feasible) and only then seeks funding. The vision is still ambitious, but the pitch is grounded in a clear adjacent possible, not a distant dream.

Art and design intuitively respect non‑objective exploration. Movements evolve through chains of influence without anyone planning their destination. Hard constraints (safety, gravity, weather) will always shape what’s possible without prescribing specific goals.

Practical Advice: keep objectives for routine, one‑step‑away tasks, but treat ambitious goals with suspicion. For exploration and innovation:

  • Favour interestingness, novelty, and rich questions over rigid targets.
  • Accumulate diverse stepping stones through varied experiences, projects, and collaborations.
  • Allow disagreement, disunity, and multiple parallel paths instead of forcing consensus.
  • Use constraints as boundaries, not destinations.
  • Judge tools and ideas by whether they open new possibilities, not just whether they “win” against current benchmarks.

The most powerful way to reach extraordinary outcomes is to stop trying to force specific extraordinary outcomes. Treat plans as provisional, follow where interesting stepping stones lead, and recognise that the biggest breakthroughs will almost always arrive from directions no objective could have justified in advance.

image

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Chapter 1: Questioning Objectives

Schools taught us the important of objectives and using test results as a measure progress. The media reframes aspirations as objectives too: lose weight, get promoted, find a partner.

But imagine a day without objectives, To-Do lists or milestones. You still do things, but just without justifying every action as progress toward a goal.

We assume that any worthy social accomplishment is best achieved by first setting it as an objective and then pursuing it together with conviction. It makes you wonder, is there such a thing as accomplishment without objectives?

The same is true at work. Engineers set specifications, then measure prototypes against them. Scientists must present clear hypotheses and objectives to secure funding. Investors target specific returns,

Even evolution in nature is described as if survival and reproduction were explicit goals.

Society seems to believe that setting an objective increases the chances of achieving it. Define a target, work hard, and you’ll get there. This makes uncertainty feel more manageable.

But that comfort comes at a cost. Objectives can make us slaves to our own measurement systems. With every activity needing to justify itself as a step toward some target. Our freedom to explore shrinks. Playful curiosity, detours, and “pointless” pursuits are crowded out because they don’t obviously move a number in the right direction.

Forcing every action to align with a preconceived outcome - often isn’t the best way to achieve great outcomes.

The typical objective mindset is: define the target, pour energy into achieving it → measure progress as you go.

Measurements give us some gradient signal, as to whether things are getting better or worse with respect to the objective.

The process of setting an objective, attempting to achieve it, and measuring progress along the way has become the primary route to achievement in our culture.

This works well for modest objectives. Become 5% more efficient, or build the next version of a software product. Because objectives work for the mundane, we start to believe they must be the right way to approach everything.

Ambitious objectives are different though → the most transformative goals are deeply uncertain. We don’t know whether they’re possible, let alone how to get there. Treating them the same way invites trouble.

A more realistic view treats achievement as discovery. Any creative act can be seen as searching through a vast space of possibilities for something valuable. Imagine a “great room” filled with all possible paintings, inventions, or ideas. Most of what fills the room is noise or junk; a few regions contain recognisably useful or beautiful things. When we create, we move through this room, guided partly by what we’ve already seen and done.

Past experiences become stepping stones. What you’ve encountered limits and shapes what you’re likely to invent next. If you’ve never encountered a region of the search space, you can’t simply will a tangential objective into being. The path depends on where you’re starting and which stepping stones you’ve already discovered.

Progress happens by moving from one stepping stone to another.

Here lies the central paradox:

Objectives are well and good when they are sufficiently modest, but things get a lot more complicated when they’re more ambitious. In fact, objectives actually become obstacles towards more exciting achievements, like those involving discovery, creativity, invention, or innovation—or even achieving true happiness. In other words (and here is the paradox), the greatest achievements become less likely when they are made objectives. Not only that, but this paradox leads to a very strange conclusion—if the paradox is really true then the best way to achieve”

Stepping stones to ambitious outcomes tend to look strange and unrelated to the final goal. The search space (the great room of all possible things) is arranged in ways we can’t easily predict.

Examples of stepping stones not resemble the final products:

  • Vacuum tubes became a key stepping stone to early computers, yet the people working on vacuum tubes were exploring electricity and radio, not computation.
  • Engines were not invented with airplanes in mind, yet airplanes depended on them.
  • Microwave ovens emerged from radar research, not from people trying to cook with radiation.

If you wanted to build a flying machine, you wouldn’t spend the next few decades instead trying to invent an engine. The paradox is that the key stepping stones were perfected only by people without the ultimate objective of building microwaves, airplanes, or computers.

The structure of the search space (the great room of all possible things) is just plain weird. It’s so bad that the objective can actually distract you from its own stepping stones! The problem is that ambitious objectives are often deceptive. They dangle a false promise of achievement if we pursue them purposefully. But strangely in the end we often must give them up ever to have the chance of reaching them.

Metrics can deepen the problem. If you mistake test scores for learning, you might optimise test prep instead of genuine understanding.

It often turns out that the measure of success—which tells us whether we are moving in the right direction—is deceptive because it’s blind to the true stepping stones that must be crossed.

The point is not to romanticise aimlessness or deny the comfort that objectives and metrics can provide. Rather, it’s to challenge the false choice between rigid goal‑seeking and random wandering. It is possible to explore a search space intelligently without treating a specific outcome as the north star. That means valuing curiosity, novelty, and genuine interest as navigational cues, instead of forcing every move to justify itself in terms of a distant, possibly misleading target.

Under this perspective, serendipity is not pure accident. The more varied and authentic the stepping stones we explore, the more likely we are to stumble onto surprising connections. Letting go of rigid objectives doesn’t mean giving up on achievement; it means recognising that the most ambitious achievements often come from paths we couldn’t have justified in advance.

Chapter 2: Victory for the Aimless

Many people design their lives around clear career objectives, yet the paths that lead to the most surprising success rarely stick to those plans. You can study one subject and end up thriving in a different field; what once seemed like the “right” objective often becomes only a stepping stone toward something better and unexpected. Because the stepping stones to fulfilling work are unknown in advance, we can’t reliably plan our way to the best outcomes.

Many successful people begin with one intention and later pivot when new possibilities appear. Their advantage isn’t a perfect plan; it’s a willingness to treat early choices as provisional and move when something better appears. Being open and flexible to opportunity is sometimes more important than knowing what you’re trying to do.

Focusing too much on your goal can actually prevent you from making useful unexpected discoveries. Serendipity is not just luck, though. Many adults trace major career decisions to chance encounters that arise from “unplanned experience”: volunteering, joining clubs, exploring new communities, or trying side projects with no clear payoff. Each new context becomes a possible stepping stone rather than a distraction.

The same dynamic extends beyond work. People choose hobbies because they are fun, yet those “pointless” pursuits sometimes become careers, artworks, or discoveries. Psychologists call the freedom behind this kind of exploration “unstructured play”: time without assigned tasks or objectives where curiosity leads. Adults need it as much as children; tinkering with tools, code, materials, or art purely for enjoyment builds capability. It’s tempting to dismiss odd passions, but they matter. They reflect that we don’t know which stepping stones might lead to something interesting. These are people who are willing to commit their lives to stepping stones that most of us would entirely ignore, which is good for all of us. Because no one knows the stepping stones that lead to the greatest discoveries, the last thing we want to do is stop people from exploring stepping stones that we ourselves choose to ignore—who knows what they will find?

You have the right to follow your passions, even when they deviate from your plans or conflict with your initial objective. Early goals can simply become stepping stones. The courage to change course can be rewarded more than any carefully managed plan. Not everything in life needs an objective justification. For the most ambitious outcomes it’s often better to treat plans as provisional and let unplanned experience, unstructured play, and genuine curiosity reveal where the next stepping stone really is.

Chapter 3: The Art of Breeding Art

The authors created a website that enabled users to breed pictures, selecting two images that would produce a generation of children. It took many generations to create recognisable shapes.

Those that focused rigidly on aiming for a specific shape struggled. Those that took advantage of interesting shapes and changed their goal created better images. The trick was not to ignore the opportunities in front of you - because you were blinded by the pursuit of a rigid goal.

Chapter 4: The False Compass

In any large, uncharted search space, stepping stones represent the waypoints that must be crossed to eventually reach the objective. In fact, the term “stepping stone” is meant to remind us of this idea of crossing a lake on small stones protruding through the water. The fundamental problem of search and discovery is that we usually don’t know the stepping stones that lead to the objective at the outset. We stand on one stone, see only a little way ahead, and still have to decide which next move is worth taking.

Ambitious problems are hard precisely because we cannot see the full path. Another way to look at the challenge of ambitious problems is to say that their solutions are more than one stepping stone away. Visionaries might spot a nearby step, but no one can reliably plan a multi-step route through a foggy landscape of possibilities.

Objective functions try to quantify progress by comparing where we are to where we want to be, but they can misfire. It’s perfectly possible that moving closer to the goal actually does not increase the value of the objective function, even if the move brings us closer to the objective. When progress is measured only by similarity to the final target, stepping stones that look different from the goal get undervalued or rejected.

This mismatch creates deception. The situation, when the objective function is a false compass, is called deception, which is a fundamental problem in search. Because stepping stones that lead to the objective may not increase the score of the objective function, objectives can be deceptive. It’s as if you were hopping across the stepping stones on the foggy lake with a broken compass. If the compass is wrong then it can deceive you, so you might never reach the other side of the lake. Deception is the key reason that objectives often don’t work to drive achievement. If the objective is deceptive, as it must be for most ambitious problems, then setting it and guiding our efforts by it offers little help in reaching it.

An alternative is to prioritise collecting promising stepping stones rather than obsessing over a single, distant objective. To arrive somewhere remarkable we must be willing to hold many paths open without knowing where they might lead. Systems like Picbreeder show that rich discovery emerges when many branches are allowed to grow without a predeclared final picture in mind.

Evolution illustrates the same principle at a far larger scale. So what should be your strategy to breed single-celled organisms all the way to human-level intelligence? Measuring each cell by how “intelligent” it appears would fail, because The problem is that the stepping stones to intelligence do not resemble intelligence at all. Key innovations like multicellularity or bilateral symmetry look nothing like the final outcome, yet are essential links in the chain.

Seen this way, evolution is not an optimiser for specific goals but a relentless stepping‑stone collector. The bottom line is that many of our greatest engineering aspirations—such as flight, solar power, artificial intelligence—were not the explicit objective of evolution, though it created all of them. It created them because nature is a stepping-stone collector, accumulating steps towards ever-more complicated novelties, marching onward onto the mist-cloaked lake of possible life-forms, heading eternally both everywhere and nowhere in particular. That’s the signature, now increasingly familiar, of processes that produce amazing innovations.

Human innovation behaves similarly. Great invention is defined by the realisation that the prerequisites are in place, laid before us by predecessors with entirely unrelated ambitions, just waiting to be combined and enhanced. The critical skill is recognising when existing stepping stones now make a leap feasible.

Objectives are not useless, but their range is limited. Maybe after all that you’re still a fan of objectives. So it’s important to recall that the concern here is with ambitious objectives, if they are only a stepping stone away, then setting and following objectives still makes good sense. The problem is that the ambitious objectives are the interesting ones, and the idea that the best way to achieve them is by ignoring them flies in the face of common intuition and conventional wisdom. More deeply it suggests that something is wrong at the heart of search. It just doesn’t seem to work like it’s supposed to. For distant, complex aims, it is often more effective to explore, accumulate diverse stepping stones, and let new combinations reveal where real progress is possible.

Chapter 5: The Interesting and the Novel

New Year’s resolutions show how easy it is to declare big objectives and how hard it is to find the stepping stones that actually lead there. Most people fail not because the goals are unworthy, but because the path is unknown and deceptive. A better use of effort is often to look for promising new experiences and directions right now instead of obsessing over a distant end state.

Objectives keep attention locked on the future: every action is judged by whether it moves you toward a goal. The problem is that distant beacons give almost no information about the real stepping stones. By contrast, the past is fully known and gives a reliable guide to novelty: you can always ask whether what you’re doing now is genuinely different from what you’ve done before. Novelty becomes a rough detector of potential stepping stones, because new things open paths to further new things.

Serendipity is often portrayed as random luck, but major “accidental” discoveries usually come from prepared, open-minded people who are actively following interesting leads. They have a nose for novelty and interestingness, even when the ultimate payoff is unclear. Their “luck” is really a systematic bias toward exploring ideas and observations that depart from the familiar.

This attitude can be codified as an algorithm with no objective: novelty search. Choose a domain and define a behaviour space. The algorithm then:

  • Generates candidate behaviours (e.g., different robot controllers).
  • Measures how behaviourally different each one is from what has been seen before.
  • Treats highly novel behaviours as “good” and stores them in a memory (an archive).
  • Mutates and recombines the most novel behaviours to generate the next candidates.

A robot driven by novelty gets “credit” for doing anything behaviourally new. At first, it crashes into a wall, that’s good, because it’s different. Then it crashes into a different wall, that’s good too, because it’s another novel outcome. After enough variations, there are no new crashes left; the only way to stay novel is to avoid walls. Eventually, for the sake of novelty alone, the robot must navigate the corridor and pass through the door. Bad behaviour (crashing) naturally precedes good behaviour (smooth navigation) without ever defining “good.”

Novelty is always relative to time and context. Once a behaviour has been seen, similar behaviours are no longer novel, so the search is forced from simple to complex: it exhausts simple actions first, then must accumulate more information about the world to keep doing new things. In evolution, the same dynamic gradually encodes facts about reality (like light and gravity) into our bodies and brains.

Sight-driven behaviour isn’t strictly necessary, it’s just that if you keep trying new designs through mutation, even though there’s no objective, eventually you will hit upon the fact that light exists. Then it will become a part of evolution’s accumulated inventory of information.

Many behaviors we can imagine are ruled out by how the world actually works. You cannot walk through walls or fly unaided just by wishing it, so entire imagined regions of behaviour space collapse to the same real outcome (hitting the wall, falling to the ground). That physical constraint quietly keeps novelty search from wandering into pure nonsense and funnels it toward behaviors that exploit real structure in the environment—those become the best stepping stones.

Natural evolution and systems are non-objective searches. Evolution is driven by a minimal constraint (survive and reproduce) plus the continual generation of new variants (diversity). Over time, this "survive + reproduce + novelty" dynamic accumulates a vast catalog of complex forms and capabilities that no one ever targeted explicitly. In that sense, evolution, human innovation, and novelty search share the same pattern: open-ended exploration that collects stepping stones rather than chasing fixed goals.

Objectives force convergence: they funnel search toward a single direction and discourage exploring paths that don’t obviously improve the objective. That means many potentially valuable regions of the search space are never visited. Divergent search instead branches into many directions at once, abandoning the comfort of heading straight for a known target. By foregoing explicit final objectives, novelty search becomes a form of divergent search, thereby joining company with natural evolution and human innovation, and aligning it with this more exotic and radical form of discovery.

In the maze experiments, novelty search repeatedly found behaviours that solved the maze even though it never “knew” about the goal. It simply kept favouring behaviours that led to new parts of the maze. Traditional objective-based search, which rewarded moving closer to the goal, was repeatedly trapped in cul‑de‑sacs that looked promising but were dead ends: classic deception. Not trying to succeed at the maze outperformed explicitly trying to solve it, because the objective function pointed the search into traps while novelty search remained free to explore.

Chapter 6: Long Live the Treasure Hunter

Not everything is possible. Some goals are physically or logically out of reach, and even when they’re possible, no search method can guarantee we’ll find the exact solution we want. Novelty search isn’t a magic shortcut either. In huge or endless spaces it can wander forever without stumbling on a particular target. Its success in mazes and walking robots shows that objective‑driven methods can be surprisingly weak, not that novelty search can always deliver what we ask for.

The deeper problem is deception. Ambitious objectives are usually deceptive: the very signal that’s supposed to guide us acts as a false compass. Hybrid schemes that mix objectives with diversity or novelty still inherit that flaw. If following the compass pushes you into walls or dead ends, then “being open‑minded” only helps when you effectively ignore the compass. At best, objectives are helpful in the tiny region where the solution is already one stepping stone away and the last move is obvious. Outside that zone, they are often a liability.

The maze experiments make this concrete. An objective‑based robot heads straight toward the goal, gets rewarded for approaching it, and so repeatedly crashes into the nearest wall pointing in that direction. The only way to make real progress is first to move away, which looks worse to the objective and so is pruned. Novelty search, by contrast, rewards doing something behaviourally new. It happily explores different crashes, then wall‑avoidance, then full maze‑traversal, and ends up solving the maze far more reliably despite never knowing there is a goal. That result, replicated by others, shows that the failure is in the objective itself, not the implementation.

Theory backs up this limitation. No Free Lunch results in optimisation show there is no “best” search algorithm across all problems: improving performance on some classes of objectives necessarily worsens it on others. There is futility at the heart of search—no rule or method can guarantee that a specific ambitious objective will be reached. If the objective is far away and heavily deceptive, neither pure objectives, pure novelty, nor hybrids can reliably get you there.

Yet search isn’t hopeless. While we can’t reliably find a particular treasure, we can reliably find some treasure. Non‑objective processes like evolution, human innovation, Picbreeder, and novelty search act as treasure hunters: they collect stepping stones without knowing where they lead, and sometimes those steps line up into something remarkable. The Picbreeder Car, for example, emerged not because anyone aimed for it, but because people kept following what looked interesting.

The practical implication is to treat discovery as opportunistic treasure hunting rather than target chasing. That means accumulating as many diverse stepping stones as possible, because no one knows which will later connect. Individually, that looks like following curiosities, building tools, and exploring ideas even when their payoff is unclear. Collectively, it looks like systems where many people branch and remix each other’s work without a single shared objective.

The interactive furniture catalog illustrates this kind of system. Instead of a fixed catalog designed by experts, customers iteratively “breed” new designs by selecting interesting variants. Over time, the site becomes a growing archive of discovered chairs—a stepping‑stone collection. Some designs will be surprising, even beyond what professionals would have conceived, precisely because amateurs explore idiosyncratic directions that no consensus would approve. Experts still matter, but their role shifts to designing the space and tools that make this exploration possible.

The key shift is to abandon the fantasy of a guaranteed path to predefined greatness and instead build processes and systems that diverge, explore, and accumulate. Search is at its most powerful when it has no unified objective: when many minds, following many different notions of “interesting,” jointly sweep the space of possibilities.

Chapter 7: Unshackling Education

Campbell’s Law suggests that the more a quantitative social indicator is used for decision-making, the more subject it will be to corruption pressures. When complex social processes are reduced to simple metrics, the metric itself becomes the target rather than the actual outcome. This phenomenon creates perverse incentives.

  • The Cobra Effect: When the British government in India offered bounties for dead cobras to reduce the population, citizens began breeding cobras to kill them for profit. The result was an increase in the snake population.

This dynamic applies to national economics through GDP fetishism. While GDP measures national productivity, maximising it as a sole objective can encourage policies that boost short-term numbers while poisoning the economy's long-term health. A simple metric cannot capture the complexity of a healthy system, yet it is often optimised to the point of deception.

In education, this objective-driven mindset manifests as high-stakes standardised testing. When teacher performance is judged by student test scores, instruction shifts to "teaching to the test." Students seek to acquire test-taking skills and memorisation rather than meaningful knowledge. The education system assumes learning is a linear climb, ignoring that the path to high achievement often requires detours that do not look like immediate progress.

The pursuit of accuracy in assessment often compounds this error. Policymakers assume that if a metric is broken, making it more accurate will fix it. However, in complex search spaces, the stepping stones to a solution rarely resemble the final solution.

In the Picbreeder experiment, the evolutionary steps required to generate an image of a skull included a crescent, a donut, and a dish. If a test had been designed to accurately measure "skull-ness" at every step, it would have rejected these necessary intermediate forms because they looked nothing like the objective. Similarly, in software engineering, early pushes for strict metrics ("You can't control what you can't measure") were later rejected by experts like Tom DeMarco, who realised that strict control and measurement are often counterproductive in complex, innovative projects.

Uniformity stifles discovery. Initiatives like the Common Core State Standards aim to equalise education by applying a single standard universally. While the intent is equality, the result is the elimination of diversity. If every school follows the same "gold standard," the entire system risks converging on the same dead end. Discovery requires a diversity of approaches, a "novelty search, where different methods are explored simultaneously. Notably, Finland’s education system significantly outperforms the US while employing no standardised tests and granting high autonomy to teachers to explore diverse methods.

The alternative to objective-based optimisation is the Treasure Hunter model: collecting interesting stepping stones without a fixed destination. Steve Jobs, for example, dropped out of college to audit classes that interested him rather than following a fixed curriculum, a strategy that provided the diverse inputs necessary for his later innovations.

To apply this to education, systems can replace standardised testing with peer-reviewed exploration:

  1. Portfolio Creation: Teachers compile portfolios of syllabi, teaching philosophy, and student work samples.
  2. Peer Review: Anonymous panels of teachers from different regions assess these portfolios on measures like innovation and curriculum completeness.
  3. Knowledge Transfer: Teachers receive detailed feedback and exposure to diverse approaches used by peers.

This method shifts the focus from "where we want to be" (the objective) to "where we are" (the stepping stones). By assessing current reality rather than forcing convergence toward a distant metric, practitioners foster the diversity and exploration required to solve complex problems.

Chapter 8: Unchaining Innovation

Magellan’s voyages and polar expeditions are reminders that the age of physical exploration has mostly ended, but the unknowns in the space of ideas remain vast. Modern explorers are scientists, entrepreneurs, and artists, and their work reshapes society as dramatically as any new continent. Yet exploration is increasingly constrained by the myth that clear objectives and consensus are the best way to make progress.

Scientific funding is a prime example. Proposals are rated by expert panels, and those with the highest average scores are funded. This rewards consensus: the safest projects that everyone can agree are “excellent.” But consensus closes off unusual stepping stones. Truly novel ideas are more likely to split the panel, some experts excited, others skeptical. When the system punishes disagreement, it selects faded compromises instead of bold, divergent paths.

The same objective thinking shows up in grand, targeted programs and in judging research by predicted impact. Projects that promise direct progress on big goals (e.g.curing cancer) seem attractive, but the real stepping stones to breakthroughs are usually unpredictable and often look trivial or irrelevant at the time. Pure mathematics done for “useless” reasons later underpins cryptography, physics, and chemistry. “Importance” becomes another deceptive objective: moderate impact today is no guarantee of radical impact tomorrow.

Funding aligned narrowly with national or economic objectives amplifies this problem. Demanding that projects clearly advance prosperity, security, or commercial outcomes assumes we can foresee where transformative ideas will originate. In reality, science progresses through serendipitous connections among many modest, curiosity-driven efforts. Over‑optimising for near‑term usefulness sacrifices the weird, indirect paths that often lead to major technologies and industries.

Not every project needs a sharp objective or testable hypothesis. Some work should be funded simply because it is interesting and opens new questions, even if its destination is unknown. One practical approach is to back people with a consistent track record of interesting discoveries, as in no‑strings fellowships, and let them roam. Forcing every proposal to promise realistic success encourages researchers to aim only for safe, one‑step‑away results—exactly the opposite of what ambitious innovation needs.

Business investing quietly acknowledges these dynamics. Investors avoid far‑future objectives like “invent holographic TV” and prefer near, concrete stepping stones like “improve display resolution using existing tech.” The real entrepreneurial trick is to discover an unexpected nearby stepping stone—like combining commodity laptop batteries into a practical electric sports car—then seek funding. The aim isn’t to promise a distant revolution, but to realize that the revolution has become just one step away.

Art and design already operate closer to the non‑objective ideal. Artists follow concepts and intuitions more than concrete goals; movements evolve through chains of influence (e.g.Impressionism to Expressionism to Surrealism) without anyone planning the destination. In architecture, hard requirements like “keep the rain out” act as constraints rather than objectives, much like “survive and reproduce” in evolution: they define the bounds, not the path within them. Yet even art students feel pressure to justify their work with explicit purposes, showing how deeply objective thinking has seeped into culture.

The broader lesson is not to discard objectives entirely, but to see their limits. They are useful for nearby, well‑understood steps, and dangerous for distant, ambitious aims where stepping stones are unknown and deceptive. For long‑range innovation, better strategies include:

  • Supporting research and projects for their interestingness, not just predicted impact.
  • Reserving some resources for ideas that split expert opinion rather than only those that win consensus.
  • Treating constraints (safety, functionality, ethics) as boundaries, while leaving goals within those boundaries open‑ended.
  • Thinking in terms of accumulating diverse stepping stones, trusting that revolutions emerge from long chains of small, surprising moves.

Seen this way, non‑objective, divergent exploration is not aimless; it is a disciplined form of treasure hunting. We can’t guarantee arrival at any specific destination, but we can reliably uncover unexpected value, provided we stop demanding that every step justify itself by pointing straight at a predefined end.

Chapter 9: Farewell to the Mirage

Abandoning objectives does not imply wandering aimlessly or living randomly. It requires distinguishing between modest objectives and ambitious objectives. For routine tasks objectives remain necessary and effective. However, when the goal is radical innovation, deep insight, or discovery, the objective compass becomes a deceiver. In complex search spaces, an ambitious target often misleads rather than guides, acting as a mirage of control we never truly possessed.

To navigate complexity without a fixed destination, one must adopt the mindset of a Treasure Hunter. The goal shifts from reaching a specific endpoint to collecting "stepping stones". intermediate discoveries that open up new possibilities. The guiding principle for this search is interestingness. While robots may rely on pure novelty to navigate, humans possess refined instincts and tastes. Following what feels "interesting" is a valid, principled heuristic because it utilises human intuition to identify high-potential stepping stones, even when the ultimate destination is unknown.

True innovators do not peer into the distant future to force a vision into reality; they look at the immediate "adjacent possible." They ask, "Where can we get from here?" rather than "How do we get there?" Innovation occurs when existing capabilities are combined or altered to take a single step forward.

  • The Minecraft Principle: Groundbreaking products often arise from combining existing concepts (e.g., melding mechanics from three previous games) rather than striving for a specific, high-fidelity future vision.
  • The Apple Approach: Success often comes from executing what is currently possible with available technology (like the iPad) rather than chasing sci-fi impossibilities (like human-level AI).
  • The Wright Brothers: While rivals sought to "claim the sky" with massive funding, the Wright Brothers treated airplanes as "bicycles in the sky," using their current knowledge as a direct stepping stone to flight.

This approach requires abandoning the notion of a single "right path." In a world of unknown possibilities, consensus is dangerous because it forces everyone onto the same stepping stone. Divergence and disagreement are virtues in discovery; society benefits when individuals pursue different interesting paths, as one person's detour may provide the necessary stepping stone for another's breakthrough.

Ultimately, the most effective way to achieve a great outcome is to stop trying to force it. If a path changes from programming to filmmaking, or from painting to poetry, it is not necessarily a failure of focus but a successful navigation of interestingness. To achieve the highest goals, we must be willing to abandon them, trusting that the accumulation of valuable stepping stones will lead to discoveries greater than any we could have planned.

Case Study 1: Reinterpreting Natural Evolution

Conventional views of natural evolution emphasize "survival of the fittest," implying a global competition to reach an optimal state of reproductive efficiency. However, this case study reinterprets evolution not as an objective-driven search for perfection, but as a minimal criteria search. In this model, survival and reproduction are not goals to be maximised, but simple binary constraints, an organism either passes the threshold to reproduce or it does not. Consequently, evolution behaves less like a strategic engineer and more like "spilled milk," passively flowing into every available niche that satisfies basic survival requirements. This aimless drift allows for exaptation (repurposing traits, like dinosaur feathers evolving for warmth before flight) and complexity, functioning like a Rube Goldberg machine that accomplishes the simple task of reproduction in increasingly elaborate ways.

The practical lesson from this biological reinterpretation is that innovation is driven by escaping competition rather than winning it. Global competition forces convergence toward a single metric of success, which eliminates diversity and leads to dead ends; in contrast, local competition encourages the founding of new niches where unique ideas can survive without being measured against a dominant standard. By treating performance metrics as loose constraints (minimal criteria) rather than strict targets to be maximied, practitioners allow for the accumulation of diverse "stepping stones." This diversity is essential because, much like in nature, the features required for a future breakthrough often evolve in a completely different context, protected from the pressure to be immediately efficient or "optimal."

Case Study 2: Objectives and the Quest for AI

The AI research community provides a revealing case study because its researchers are experts in search algorithms, yet the community's own search for new algorithms is guided by two objective-driven heuristics.

  • The experimentalist heuristic filters ideas based on whether they outperform existing algorithms on benchmarks.
  • The theoretical heuristic filters based on whether algorithms come with mathematical guarantees.

Both act as gatekeepers: if your novel algorithm performs 5% worse than the status quo, or lacks proven properties, it will struggle to get published regardless of how interesting or potentially generative it might be. The result is that entire branches of the search space get pruned away because they're only reachable through algorithms that don't pass these filters. The key lesson is that what helps a practitioner choose today's best tool is different from what helps a researcher find tomorrow's breakthrough. Performance comparisons between distant points in a vast space tell you nothing about which direction to explore. An algorithm should be judged not by whether it beats current benchmarks, but by whether it leads researchers to think of other algorithms, whether it opens new stepping stones. The same principle applies to any community filtering ideas: rigid objective heuristics become excuses not to engage with substance, making it easier to dismiss unfamiliar ideas than to consider what they might make possible. Real expertise means thinking with an open mind about inspiration, elegance, novelty, and potential to provoke further creativity—not demanding that every new idea first win a race against the current favourite.