The Bitter Lesson

The Bitter Lesson

Rich Sutton. 2019. (View Paper → )

In AI development, approaches that leverage raw computational power consistently outperform systems built with human expertise baked in. This counterintuitive pattern has repeated across chess, Go, speech recognition, and computer vision, where statistical methods analysing massive amounts of data beat carefully crafted expert systems. The lesson is "bitter" because it challenges our belief that human intelligence is special and that programming our knowledge into AI is the path forward. Instead, Sutton suggests we should focus on creating systems that can learn and discover knowledge on their own rather than trying to encode our discoveries, implying that general methods scaling with computational power are ultimately more effective than those trying to mimic human thinking.