Researchers trained a 1.14 billion parameter Learner 1.0 model on ten unrelated skills sequentially, one example at a time with no replay, demonstrating that later skills did not erase earlier ones. Final evaluation showed minimal performance degradation across retained skills, with most skills maintaining their learned scores after all ten sequential trainings.
Researchers introduce long-horizon memorization, a challenge where language models must learn 100 tasks through continual fine-tuning without catastrophic forgetting. They show that composing multiple continual learning mechanisms—combining data, function, and weight anchors with merged LoRA—improves retention from 1.2% to 34.9%, a 28-fold improvement over naive sequential fine-tuning.