Universal Intelligence: A Definition of Machine Intelligence
Shane LeggMarcus Hutter
Develops a mathematically rigorous definition of intelligence that applies to any agent—biological or artificial—by formalizing the intuition that intelligence means succeeding at a wide variety of tasks weighted by their complexity.
A fundamental challenge in artificial intelligence is defining and measuring intelligence for machines that may operate very differently from humans. Traditional approaches, like IQ tests for people or the Turing test for computers, often rely on human-like traits such as language or reasoning in familiar contexts. These fall short for diverse systems, from robots to algorithms, especially as technology advances and creates entities with novel capabilities. Without a clear, general definition, it's hard to gauge progress in AI or compare systems objectively, which hinders decisions on investment, development, and risks like unintended behaviors in superintelligent machines.
This paper sets out to create a formal, universal definition of machine intelligence by drawing from established ideas on human intelligence. The authors review psychological theories, tests, and expert definitions to identify core elements—such as adapting to new situations, learning from experience, and achieving goals—and translate them into a mathematical framework suitable for any machine.
The approach starts with a simple model: an agent interacts with an environment by taking actions, receiving observations and rewards, much like how animals learn through trial and feedback. To capture generality, the authors consider all possible computable environments—simulated scenarios drawn from an infinite but describable set—weighted by their simplicity, using a concept from information theory called Kolmogorov complexity (essentially, the shortest program needed to describe something). This favors testing against straightforward yet varied challenges, reflecting the principle that simpler explanations are often best, as seen in human IQ tests like pattern recognition. They then define universal intelligence as an agent's expected success (total rewards) across these environments, without assuming specific hardware, senses, or goals.
The key results include a precise equation for universal intelligence, denoted Υ, which ranks agents logically: a random actor scores near zero, basic learners that track patterns do better, specialized systems like a chess computer falter on unfamiliar tasks, and the theoretical optimal agent (AIXI) achieves the maximum. The paper also surveys other machine intelligence proposals, from Turing test variants to compression benchmarks, and compares them favorably against criteria like generality and objectivity—universal intelligence stands out for its breadth and lack of human bias. Notably, it aligns with proven optimal learning theories, showing that highly intelligent agents excel in prediction, planning, and adaptation across domains.
These findings imply a robust way to think about machine intelligence as the ability to thrive in diverse, unpredictable settings, impacting costs by guiding efficient AI design, reducing risks through better evaluation of adaptability, and informing policy on ethical AI deployment. Unlike narrower tests, this definition avoids cultural or species biases, evolving with technology rather than tying intelligence to human norms. It challenges views that equate intelligence with efficiency or consciousness, focusing instead on measurable performance.
Next steps should involve building practical tests to approximate Υ, such as sampling environments by generating short programs and running agent simulations, then weighting results by program length. Pilot these on existing AI systems to validate rankings against real-world utility. Trade-offs include balancing test scope (more environments mean higher accuracy but longer computation) with feasibility. Further analysis could explore how human cognition fits this scale.
Limitations include the definition's reliance on computable environments, which assumes the universe follows Turing-machine rules (no evidence against this yet, but unproven), and the uncomputability of exact complexity measures, requiring approximations that might introduce errors. Confidence is high in the theoretical foundation—rooted in decades of work on learning and complexity—but lower for immediate applications; readers should be cautious about over-relying on it without empirical validation through tests. Overall, this work provides a foundational tool for assessing AI's potential and progress.
- Book: Machine Super Intelligence, Shane Legg. Its formal treatment of universal intelligence, algorithmic probability, and AIXI supplies the theoretical framework this paper distills into a definition and measure.
- Paper: Position: Levels of AGI for Operationalizing Progress on the Path to AGI, Meredith Ringel Morris et al. (2024). It turns broad questions about general intelligence into a capability-and-performance framework, offering a later operational approach to classifying progress beyond a universal measure.
