Sutton: Raw Compute Insufficient for True AI Intelligence
Turing Award winner Richard Sutton argues that AI requires new methods beyond scaling, sparking debate among industry leaders.
Turing Award winner Richard Sutton has stated that raw compute alone is not enough for intelligence. He argues that AI science requires clever new methods and is far from finished. Sutton, who authored the influential 2019 essay 'The Bitter Lesson,' previously criticized the limits of simply scaling large language models. He called synthetic data a 'big mistake' and argues that human expertise acts as a bottleneck preventing true scaling. Sutton notes that today's language models do not keep learning after training. He argues that the real world is infinitely complex and simulations are only microscopic. Sutton calls for agents with world models that continuously learn from their own experience.
Jerry Tworek, who runs the AI lab Core Automation, states that the bitter lesson still holds but algorithms matter more as compute increases. Tworek claims that a misreading of the bitter lesson has reward-hacked the brains of a generation of scientists. Jayden Teoh argues that the bitter lesson has held back academic research by dismissing promising ideas. Teoh argues that an idea does not necessarily have to survive scaling forever to be an important contribution.
OpenAI CEO Sam Altman considers large language model scaling far from exhausted. Altman accuses LLM critics of slowing down an entire generation of researchers. He points to an OpenAI model that recently disproved a mathematical conjecture as evidence for scaling. Anthropic CEO Dario Amodei argues that scaling progress is like a rainbow with no end. Sutton runs Oak Lab alongside other founders who doubt today's AI recipe alone will succeed.
Sources
Every sentence of this story was checked against facts quoted from these articles. How we make the news
Spotted an error? Tell us
