A machine learning study challenges the conventional wisdom that data filtering is essential for large model pretraining, finding that with sufficient computational resources, unfiltered data—including low-quality information—can be as effective or beneficial as curated datasets.
A reverse-engineered model mimics Jev, TypeSafe's commercial system for selecting from multiple text options in a single pass. The repository includes implementations for Doom and chess games, with training and evaluation tools using attention-based scoring across option-context pairs.
MetaRSI-v1 is a recursive self-improvement system that extends beyond formal benchmarks to operate across diverse scientific and engineering domains by composing three typed operators—Data-RSI, Harness-RSI, and Model-RSI—over a unified paradigm, enabling both parameter-level and interface-level improvements without external supervision.