Researchers propose a recurrent language modeling approach using balanced-tree recursive operators that achieves logarithmic computational depth and linear runtime, offering an alternative to Transformers with fixed depth and quadratic complexity. Experiments show the model extrapolates well to longer sequences and approaches Transformer performance with ALiBi.
LatentPort demonstrates cross-model transfer of recurrent inference state from a 4B to 9B Qwen language model without replaying the source context, using hybrid-state handoff combining translated attention KV cache with Gated DeltaNet persistent-state components. The approach achieves near-native performance with only a 0.076 nats/token excess loss on continuation tasks.