Engram extends token embeddings with learned multi-token lookups to reduce computational overhead in large language models. The architecture enables efficient memory offloading from HBM to DRAM and SSD, allowing larger models to run on constrained hardware. Researchers validated Engram across multiple GPU architectures and inference frameworks, finding that offloading embeddings to DRAM can improve performance even on high-capacity systems.