Giant AI models don't always overfit despite having capacity to memorize training data because data structure matters more than model size, and training algorithms have implicit biases that favor simpler solutions. The double descent phenomenon shows test error can improve again after initial overfitting, while the distribution of noise across weak feature directions can preserve predictive performance on unseen examples.
DreamZero is a World Action Model that learns robot control by jointly predicting future world states and actions using video diffusion. It achieves over 2× better generalization to new tasks and environments compared to Vision-Language-Action models, and can adapt to new robot embodiments with just 30 minutes of play data while maintaining zero-shot generalization capabilities.
RRSI is a method for evolving AI agent harnesses that avoids overfitting to training benchmarks through regularized search constraints. The approach improves performance on held-out benchmarks by controlling edit magnitude, requiring measured gains to exceed variance, and eliminating benchmark-specific logic, with all candidates logged for transparency.