Measuring AGI: New Benchmarks and Challenges

Evaluating AGI is challenging due to inconsistent definitions of human-level intelligence and issues like data contamination. Traditional metrics, such as the Turing test, no longer suffice, prompting emerging benchmarks like ARC-AGI that aim to test abstraction and generalization. Large language models exhibit low scaling gains, making standard metrics unreliable. Experts advocate frequent benchmark updates and robust evaluation frameworks to better assess progress toward AGI.