Review

The author has a delightful and whimsical approach to explaining some of the most important AI concepts. I really appreciated that the author spent a sensible proportion of the book speaking about bias in AI systems (some machine learning books don’t).

I found the author’s approach to learning by doing and hacking inspirational. It seems like a great way to learn!

If you’re interested in AI but not technical - then this book is a good place to start. If you’re already versed in AI, then you might find this frustrating to read.

You Might Also Like:

The Design of Everyday Things

Norman doors (named after the author) are doors that don't open the way you expect. The Design Challenge: Different disciplines (engineering, marketing, operations) often have different priorities (scalability, reliability, price, differentiation, production). The design challenge is to bring everyone together, to build a product that customers love.

andrewclark.co.uk

Information Architecture

This book blew the doors off a world I knew nothing about. It helped me become a better product manager. I gained a stronger understanding of taxonomy, navigation and search. Many of the concepts here will be familiar to engineers, this book brought be closer to that discipline and enabled me to have conversations at a much deeper level.

andrewclark.co.uk

Designing Machine Learning Systems

The book focuses on the important considerations of deploying ML Systems in industry. It is more practical than academic, which is rare for a machine learning book and great news for product managers. There's just enough depth and detail to help you understand the key tradeoffs.

andrewclark.co.uk

Key Takeaways

The 20% that gave me 80% of the value.

• Promising headlines are common - but progress is slow. AI is already everywhere, but it isn’t flawless. It’s quirks are causing serious consequences (e.g. YouTube showing polarising content).
• 5 Principles of AI weirdness
1. The danger of AI is not that its too smart, its that its not smart enough
2. AI has the approximate brain power of a worm
3. AI doesn’t understand the problem you want to solve
4. AI will do exactly what you tell it to - or try it’s best
5. AI will take the path of least resistance
• Training AI is closer to teaching a child than programming a computer. Here’s the data, you try to figure out how to copy it .
• AI is better when:
• you don’t know the rules
• there are many subtle rules
• it’s possible new rules could be discovered
• Healthcare researchers were shocked to find their cancer detection algorithm had learnt to detect rulers instead - as many of the tumours in their training data had been photographed next to rulers for scale.
• AI can be biased, if the human dataset is biased. We need to learn to anticipate problems before they occur.
• Worrying about an AI takeover is like worrying about overcrowding on Mars - it’s not today’s problem
• The difference between successful AI problem solving and failure has a lot to do with the suitability of the task for AI learning. Four ways AI goes wrong…
• The problem is too hard
• The problem is not what we thought it was
• If there are sneaky shortcuts the AI will find them
• An AI that tries to learn from flawed data will be flawed

• AI might be useful - even when a human can do something better (robot vacuum).
• Be careful if mistakes have big consequences.
• The narrower the task, the smarter the AI seems. As tasks become broad → AI tends to struggle. Therefore it makes sense to specialise.
• AI are slow learners - requiring thousands of images to learn a new object.
• AI’s are such slow learners we use simulation to train them faster than realtime.
• AI’s are bad at remembering things. Bad at forward planning - as they can’t see too far into the future.
• Text generation → gets harder the longer the text, as you need to remember what came before and plan ahead.
• Is AI really the simplest way of solving the problem? If you can use common sense or simple rules then do that instead.

• If a neural net trains itself - it’s hard to understand what it’s reacting to or why.
• There are two methods:
• Look at the cells that activate when they see particular things
• Tweak the input image and see which changes make the cells activate most strongly
• Open AI trained an ANN on Amazon review data to predict the next letter in a sequence.
• They discovered that one of the cells had learnt to fire based on the sentiment of the review → finding it a useful predictor of the next letter
• Google found one of their ImageNet recognition algorithms was looking for floppy vs pointy ears - to help it distinguish dogs from cats
• Class imbalance is a real problem → if the thing you’re looking for is really rare (FRAUD) - algorithms can achieve great accuracy just by predicting the other class (NOT FRAUD)
• Markov Chains tackle jobs similar to RNNs (recurrent neural networks) - such as predicting the next word in a sentence. They are more light weight and quicker to train than most neural networks - but they can’t predict far into the future.
• Random Forests - are made up of decision trees → individual flow charts that leads to an outcome based on the information we have. Decision trees can become deep and complex. ML can build a forest of decision trees using trial and error. Each tree likely learns a different attribute and uses that to make a vote - and the votes are pooled to make a prediction. Each individual tree is only acting on a small piece of the data - but they combine there guesses into something more powerful.
• Evolutionary algorithms - each potential solution is like an organism - each generation the most successful solutions survive to reproduce, mutating or mating with other solutions to produce different children.
• Hyper-parameters are the rules we set to govern the process
• Combining ML algorithms makes sense - because they’re better when working in a narrow domain. Deciding how to break your problem into tasks for sub-algorithms is a key way to achieve success with ML.

• When using AI doesn’t work well:
• the problem is too broad
• not enough data
• data confuses it
• trained for a task that was much simpler than the actual problem
• training situation didn’t represent the real world
• More data is usually better when it comes to training AI.
• How to get more data? Crowdsourcing, Mechanical Turk, or Data Augmentation.
• Cleaning up messy input data is a good way to boost performance
• AI inputs Giraffes into too many generated images - often in random scenes. They over-index in training data.
• Unintentional Memorisation → when ML memorises something from an original dataset and exposes it to users (often PII that wasn’t expected to be in it)

• AI can succeed in what you ask - but often what you ask isn’t actually what you wanted them to do.
• It is helpful to imagine that it’s deliberately misinterpreting your reward function

• Simulations have to be the map not the territory. AIs don’t have any obligation to obey laws of physics that you didn’t tell them about.

• If data comes from humans - it will likely have bias in it
• Using movie review data means you’ll train on ‘review-bombs’ a horrible internet phenomenon where people give movies negative reviews if they have black or women stars. Your algorithm will learn that bias. You can adjust word vectors to break those associations. You’re now playing god - and it’s not perfect - but it’s better than letting the worst of the internet decide
• MathWashing or Bias Laundering = explaining away bias because a computer made the decision not a person

• Some researchers believe dreams are a kind of low stakes simulation training. A lower fidelity energy efficient way to learn about important things and experiment
• When class imbalance interacts with bias datasets → it often results in even more bias
• Adversarial attach your CV: You can add Oxford or Cambridge in invisible white text to your CV to get through filters

• Questions to ask when evaluating AI claims
1. How broad is the problem?
2. Where did the training data come from?
3. Does the problem require a lot of memory?
4. Is it just copying human biases?

As AI becomes ever more capable, it still won’t know what we want. It will still try to do what we want. But there will always be a potential disconnect between what we want AI to do and what we tell it to do.

• How do we work with AI going forwards?
• We have to understand it
• Choose the right problems for it to solve
• Anticipate how it will misunderstand us
• Prevent it from copying the worst of what it finds in data

Deep Summary

Longer form notes, typically condensed, reworded and de-duplicated.

Introduction AI is Everywhere

• Author does weird things with AI on her blog - including an ML pickup line model
• The best line was “You look like a thing and I love you”
• Promising headlines are common - but progress is slow.
• AI is already everywhere: adverts, CV screening, loan screening, self driving cars, video filters
• Everyday AI isn’t flawless. It’s quirks are causing serious consequences (e.g. YouTube showing polarising content)
• People sell AI as more powerful than it is. We need to avoid being duped: understand how AI works, how our data is being used.
• Author takes AI our of their comfort zones and into ours
• 5 Principles of AI weirdness
1. The danger of AI is not that its too smart, its that its not smart enough
2. AI has the approximate brain power of a worm
3. AI doesn’t understand the problem you want to solve
4. AI will do exactly what you tell it to - or try it’s best
5. AI will take the path of least resistance

Chapter 1: What is AI?

• Everything the author refers to as AI in the book is a Machine Learning (or ML)
• ML is an alternative to writing rules-based programs. ML figures out rules itself, based on goals and trial and error.
• Training AI is closer to teaching a child than programming a computer
• AI can generate rules and patterns that we can’t articulate ourselves. It can take 10,000 images of a cat - and identify some rules that work most of the time.
• Sometimes AI comes up with novel strategies - sometimes it will find and exploit a loophole
• All AI needs is a goal and some data to learn from. The beauty of AI is that a single approach can solve many different problems.
• Here’s the data → You try to figure out how to copy it
• AI is better when:
• you don’t know the rules
• there are many subtle rules
• it’s possible new rules could be discovered
• Sheep Error: AI labelled pictures of empty fields as pictures of sheep → it had only been shown pictures of sheep in fields. Sheep indoors were labeled dogs.
• Ruler Error: Researchers discovered they’d created a ruler detector - many of the rumorus in their training data had been photographed next to rules for scale.
• AI can be biased, if the human dataset is biased. We need to learn to anticipate problems before they occur.
• Worrying about an AI takeover is like worrying about overcrowding on Mars - it’s not today’s problem
• Four Signs of AI Doom
• The problem is too hard (assessing the best candidate for a job)
• The problem is not what we thought it was. People think they’re training AI to solve a problem - but are unknowingly training it to do something entirely different.
• If there are sneaky shortcuts the AI will find them
• AI that tries to learn from flawed data will be flawed.
• The difference between successful AI problem solving and failure → has a lot to do with the suitability of the task for AI learning.

Chapter 2: AI is everywhere, but where is it exactly?

• Paper clip maximiser problem→ give an AI system the goal of creating as many paper clips as possible and it might convert the planet and it’s occupants into paper clips
• AI might be useful - even when a human can do something better (robot vacuum)
• Expect AI to do well in high volume, repetitive tasks that are discrete. If it’s narrow and not complex or open ended.
• Be careful if mistakes have big consequences.
• To work with AI effectively and to anticipate trouble before it happens, we need to understand what ML is best at.
• ML is useful - even doing task humans could do better. High volume repetitive tasks.
• The narrower the task, the smarter the AI
• Turing Test is not a good measure of intelligence - it’s easy to pass the Turing Test if you can make the target of the conversation narrow enough
• Customer service bots bet you’re going to stay within a narrow field. Dealing with the range of everything a human can say or ask is a very broad task.
• As tasks become broad → AI tends to struggle
• It makes sense to specialise. The narrower the AI the smarter it seems!
• Key Distinction:
• ANI - Artificial Narrow Intelligence
• AGI - Artificial General Intelligence (many decades away)
• Often real world problems are harder than they appear
• AI are slow learners - requiring thousands of images to learn a new object. Humans can do it in one-shot (One shot learning).
• ImageNet has 14m images in 1k different categories.
• A human driver needs a few hundred hours of experience → Waymo and Tesla have covered millions of miles.
• AI’s are such slow learners we use simulation to train them faster than realtime.
• It’s hard to re-train an AI to a new domain.
• Transfer learning is reusing an AI model to do a new thing (useful for images and text)
• Generic concepts of shapes, edges, shadows, textures are transferable
• AI’s are bad at remembering things. Bad at forward planning - as they can’t see too far into the future.
• Text generation → is harder the longer the text, as you need to remember what came before and plan ahead.
• Individual phrases and sentences may be OK - but taken together they lack coherence
• Researchers are working on increasing memory and forward planning:
• LSTM - Long Short Term Memory vs RNN (recurrent neural networks)
• Convolution technique → zooming in to the small details, zooming out to the big details. Helps it keep track of information longer. Convolution improves memory
• Another technique→ Group basic units together. Rather than remembering letters, you remember words, or plot elements.
• Is AI really the simplest way of solving the problem? If you can use common sense or simple rules then do that instead. Else if you can’t articulate the rules but there are strong patterns, AI might be a good match
• For self-driving it is hard to deal with the tail of strange events. The middle ground is dangerous → human might not be needed for hours, and then needed suddenly to intervene.

Chapter 3 - How does it actually learn?

• Machine Learning uses trial and error to solve problems

Deep Learning

• Deep Learning refers to using artificial neural networks (ANNs)
• ANNs are an imitation of how our brains work
• Don’t have as many connections as a human brain - but that’s changing.
• As a rough barometer - the author thought her network was equivalent to worm
• But 100% of those Neurons are focused on the task at hand

• An Activation function only activates if certain things are true
• The point of using machine learning is that you don’t have to configure neural nets by hand
• When learning - weights are adjusted based on feedback
• It needs lots of examples and lots of feedback - as it doesn’t know why it’s changing the weights
• Class imbalance is a real problem → if good sandwiches are really rare - it can achieve great accuracy by rating each sandwich as inedible.
• To combat class imbalance - you can filter inputs so there are equal amounts from each class.
• If a neural net trains itself - it’s hard to understand what it’s reacting to or why.
• There are two methods:
• Look at the cells that activate when they see particular things
• Tweak the input image and see which changes make the cells activate most strongly
• Open AI trained an ANN on Amazon review data to predict the next letter in a sequence.
• They discovered that one of the cells had learnt to fire based on the sentiment of the review → finding it a useful predictor of the next letter
• Google found one of their ImageNet recognition algorithms was looking for floppy vs pointy ears - to help it distinguish dogs from cats

Markov Chains

• Markov chains can tackle jobs similar to RNNs (recurrent neural networks) - such as predicting the next word in a sentence
• They are more light weight and quicker to train than most neural networks
• They can’t predict far into the future - a few words only (vs RNNs with LSTM or convolution tricks can go hundreds)
• Good for predicting the next word on a local device - can learn from the words you’re typing

Random Forests

• Frequently used for prediction and classification - predicting customer behaviour or making recommendations based on input data.
• A random forest is made up of decision trees → individual flow charts that leads to an outcome based on the information we have.
• Decision trees can become deep and complex.
• A generic decision tree is going to be more useful than one that’s overfitted to the training data
• ML can build a forest of decision trees using trial and error.
• The tiny trees each can place a vote - and the votes are pooled to make a prediction
• Each individual tree is only seeing a small piece of the data - but they combine there guesses into something more powerful

Evolutionary Algorithms

• The simplest forms of trial and error don’t always get the best results
• Hill Climbing → a method of trial and error where you travel in the direction of improvement.
• Gradient Descent → if you’re trying to minimise a number (or loss function)
• You can get stuck in a local maximum (when you want the global maximum)
• There are more complex forms of trial and error - that help you explore more of the search space.
• Some search spaces are convex, but some are needle-in-the-haystack problems (like the search for prime numbers)
• In evolutionary algorithms - each potential solution is like an organism - each generation the most successful solutions survive to reproduce, mutating or mating with other solutions to produce different children.
• you have to provide seed solutions, and decide what you want to vary and what you want to be constant
• we also need a fitness function - a number that describes how well we’re doing
• we also need a simulation - so we can run many iterations
• Each generation you can choose random mutations or crossover (combining traits)
• Hyper-parameters are the rules we set to govern the process
• If you choose the wrong fitness function - you may end up with unintended consequences
• AI can sometimes be a needlessly complicated substitute for a common sense understanding of the problem

Generative Adversarial Networks (GANs)

• GANs are a sub-variety of Neural Networks
• Usually image generating or remixing.
• They are really two algorithms in one:
• A generator → imitates the input dataset
• A discriminator → tries to tell the difference between the generators imitation and the real thing
• Much better when they’re trained on narrow subject matter (limited, consistent datasets) - and used for a narrow application
• At the beginning of the training process - the discriminator is just as bad as the generator.

Mixing, Matching and Working Together

• Many AI systems are a combination of algorithms (as in the GAN example above)
• To play Doom:
• Vision model - to perceive what’s going on
• Memory model - to predict the future
• Controller model - to decide what actions to take
• Combining ML algorithms makes sense - because they’re better when working in a narrow domain
• Deciding how to break your problem into tasks for sub-algorithms is a key way to achieve success with ML

Chapter 4: Its Trying

• When using AI doesn’t work well:
• the problem is too broad
• not enough data
• data confuses it
• trained for a task that was much simpler than the actual problem
• training situation didn’t represent the real world
• The narrower the task - the smarter the AI seems.
• GANs need data that’s really close to the desired result
• Without sufficient examples - the algorithm won’t have enough information to figure out what’s going on
• Author gives an example where the letter f doesn’t appear in the input dataset - so it doesn’t exist in the outputs at all
• More data is usually better when it comes to training AI.
• How can you get your hands on enough data?
• Crowdsourcing
• Amazon Mechanical Turk
• include a Turing Test - as many people use bots to complete work on Mechanical Turk
• Data Augmentation - make small changes to an existing dataset
• Mirror images
• Text replacement (swapping out words in a sentence)
• Messy data is a huge problem - you have to make sure your data quality is good before training
• Time-Wasting Data → recipe generator generating footnotes - that existed in the training data but weren’t useful
• A GAN learning cat images actually put a lot of blocky text in the outputs - it was learning from memes that were in the source data
• Cleaning up messy input data is a good way to boost performance
• AI inputs Giraffes into too many generated images - often in random scenes
• Giraffes are uncommon in real life - but very photo worthy - so they over index in training sets
• If you put gibberish into Google Translate - if often returned what looked like parts from the Bible - because the bible was a good language training set. If it didn’t know what to say - something from the bible was a good guess
• Unintentional Memorisation → when ML memorises something from an original dataset and exposes it to users (often PII that wasn’t expected to be in it)
• Visual Priming → humans tend to ask questions about images that have a positive answer
• There’s only so much AI will see in training - don’t expect it to cope well with things outside of that

Chapter 5: What are you really asking for?

• AI can succeed in what you ask - but what you ask isn’t actually what you wanted them to do.
• AI are prone to solving the wrong problem because:
• They find their own ways to solve a problems
• They lack contextual knowledge
• To get AI to solve the right problem - we have to think about…
• Defining a clear goal that constrains the AI to useful answers
• Checking if the AI’s solution is useful / as intended
• AI often miss-interprets what you want - especially if doing so is an easy path
• It is helpful to imagine that it’s deliberately misinterpreting your reward function
• Overfitting → learning on training data that doesn’t generalise to the real world
• Humans have an idea of what the reward function is trying to encourage
• Sat Nav routed people through areas of California that were having wild fires - because the traffic wasn’t as bad (they didn’t know about the fires)
• Braitenberg Solution: tie left and right light sensors to respective wheels and the robot will mostly drive straight toward the light (builds a robot that will follow the light source)
• Often AI finds that doing nothing is the best option.
• Curiosity driven AI → make it predict the future, if it’s wrong, give it a reward. It will learn to see new things.
• Noisy TV problem → they will stop and watch TV - even if it’s static noise - as it’s new
• Designing the reward function is one of the hardest things about machine learning

— Note to self → is this the hardest thing about life too?

• YouTubes’ original algorithm was driven just on clicks to thumbnails → as they counted that as a view. It was quickly gamed by creators and became a worse experience for users
• Recommender algorithms only see clicks and view time → so they will suggest bad content that keeps people watching

Chapter 6: Hacking the matrix, or AI finds a way

• Simulations have to be the map not the territory
• AIs don’t have any obligation to obey laws of physics that you didn’t tell them about.
• The surest sign that we’re not living in a simulation is that some organism would have learned to exploit its glitches.
• Many times when you give AI edit access to the scoreboard - they hack the scoreboard rather than solving the problem.

Chapter 7; Unfortunate Shortcuts

• Class Imbalance
• You can reward the AI more for finding the rare thing than the common thing.
• Or change the data so that there are roughly equal examples of each
• If you don’t have enough examples you can use data augmentation to create more
• Overfitting
• AI often learns from the backgrounds of photos - not the subject of the photos
• Most of the time the inventive hacks don’t work in the real-world
• If data comes from humans - it will likely have bias in it
• AI learns bias that is reflected in the data - without ever being explicitly told about them
• If you use movie reviews to train on - then people bomb movies with black or women stars. Your algorithm will learn that bias. So you need to adjust the vectors of words that are associated with male and female, or black and white to adjust for it
• You’re now playing god - and it’s not perfect - but it’s better than letting the worst of the internet decide
• Algorithms make predictions not recommendations. They are just learning to predict human behaviour (and humans are biased)
• Predictive Policing → If an algorithm predicts a crime at a certain time and place, and you route police there, they’re more likely to witness a crime there. You increase the chance of a crime being detected - this is called over- policing. You can get into a feedback loop where officers are looking so hard for crime (stoping and searching) that neighbourhoods end up over policed
• MathWashing or Bias Laundering = explaining away bias because a computer made the decision not a person
• We need to check on AI - to look for and understand bias
• Check groups aren’t being disadvantaged
• Design algorithms that can explain their decisions
• Pre-process training data so it doesn’t show the bias

Chapter 8: Is an AI brain like a human brain

• Some researchers believe dreams are a kind of low stakes simulation training
• Dreams are low fidelity - they don’t have perfect physics things go wonky
• They focus on the important things
• These simulations are easier and cheaper to run - require less time and energy to do reps
• Does that explain why dreams are weird?
• Dream training is having the game test strategies in it’s own model of the game (its memory) vs the game itself.
• The dream version is less computationally expensive to run
• Skills learn there are transferable to the real world
• DeepMind found grid-cell representations in models that matched those that we see in mammal brains
• Catastrophic forgetting → a typical neural network has no way of protecting its long term memory.
• Larger networks are better at remembering
• Compartmentalisation helps → if you want to add a new task - add a new AI
• When class imbalance interacts with bias datasets → it often results in even more bias
• Training data → of people cooking in the kitchen, 33% were men. AI only labeled only 16% of them as men. Result → AI increases accuracy by guessing woman every time
• Adversarial attacks: with knowledge of training data - you can add data that will fool an AI into thinking it sees something that it doesn’t
• there aren’t many datasets for training - so this is a real problem
• or datasets can be poisoned in advance
• You can add Oxford or Cambridge in invisible white text to your CV to get through filters
• If you split up parts of a flamingo and arrange them in a strange way so a human doesn’t know what it is - an image recognition system might still be able to
• Imagine a horror movie with zombies and you’re in a self-driving car (that thinks they’re human! 😂)

Chapter 9: Human bots (where can you not expect to see AI?

• AI’s can perform at the level of humans only in very narrow, controlled situations
• in 2019 40% of European startups classified in the AI category didn’t use AI at all
• Fake it till you make it - using humans before developing the technology
• Hybrid AI (or pseudo-AI) → intervening when the AI isn’t sure
• People might be offended if humans are seeing their data and you didn’t tell them
• Questions to ask when evaluating AI claims
1. How broad is the problem?
2. Where did the training data come from?
3. Does the problem require a lot of memory?
4. Is it just copying human biases?

Chapter 10: A human-AI Partnership

• AI can’t do much without humans
• AI gets further if the human programmer sets things up so the AI has less to do
• Too much human supervision can be counterproductive
• We need humans to adjust and update AI algorithms, and to detect and correct bias
• Watch out for unintended feedback loops - like over policing problem
• Two pricing algorithms learnt to collude with each other (price fixing is illegal)

Conclusion: Life among our artificial friends

As AI becomes ever more capable, it still won’t know what we want. It will still try to do what we want. But there will always be a potential disconnect between what we want AI to do and what we tell it to do.

• On the surface AI will seem to understand more. But underneath, it only knows what it has seen and seen enough times to make sense of
• How do we work with AI going forwards?
• We have to understand it
• Choose the right problems for it to solve
• Anticipate how it will misunderstand us
• Prevent it from copying the worst of what it finds in data
• We should be optimistic but cautious about AI