Recurrent Looped Transformer (RLT) combines a causal encoder with a recurrent decoder that grows temporal depth with each token, traversing tL_D decoder blocks after t tokens while maintaining fixed per-token computation. The architecture integrates global key–value memory, layerwise sliding-window attention caches, and continuous latent computation across prompts and responses, with co-design considerations for hardware efficiency and reinforcement learning scaling.
Recurrent Looped Transformer (RLT) combines a causal encoder with a recurrent decoder that grows temporal depth with each token, traversing tL_D decoder blocks after t tokens while maintaining fixed per-token computation. The architecture integrates global encoder memory with layerwise sliding-window attention caches in the decoder, enabling co-design with hardware and RL algorithms through shared state transitions across pretraining, fine-tuning, and sampling.