A reverse-engineered model mimics Jev, TypeSafe's commercial system for selecting from multiple text options in a single pass. The repository includes implementations for Doom and chess games, with training and evaluation tools using attention-based scoring across option-context pairs.
Recurrent Looped Transformer (RLT) combines a causal encoder with a recurrent decoder that grows temporal depth with each token, traversing tL_D decoder blocks after t tokens while maintaining fixed per-token computation. The architecture integrates global encoder memory with layerwise sliding-window attention caches in the decoder, enabling co-design with hardware and RL algorithms through shared state transitions across pretraining, fine-tuning, and sampling.