A blog post discusses how reinforcement learning improvements for large language models disproportionately benefit easier problems while leaving harder ones largely unsolved—a phenomenon called the Matthew Effect. The authors propose a solution called Never Give Up to address this bias and improve performance on genuinely difficult tasks.
A study comparing AI progress from 2019 to 2025 finds that data improvements have contributed 12.0x efficiency gains versus 3.7x from model improvements, with data accounting for 3.24x more of the total gains in pretraining. The research evaluates combinations of year-representative model architectures and data corpuses across different compute scales, finding that data and model improvements are largely independent and additive.