TileRT achieved 469 tokens per second on AMD Instinct MI355X GPUs, topping the InferenceX AgentX leaderboard for inference performance on real-world coding agent workloads. The benchmark measures sustained performance across long-horizon agent tasks with growing context, where TileRT retained approximately two-thirds of its speed even at one-million-token context lengths.
MiMo-V3 introduces HySparse2, a new architecture designed for agentic inference that reduces prefill FLOPs by 5× and KV cache size by 4.5× compared to MiMo-V2.6 while improving long-context retrieval. HySparse2 uses two levels of KV sharing—KV Bridging and KV Reuse—along with token-level selection and a unified cache for local and global tokens to optimize the workload of processing short actions followed by long observations.
HySparse2 is a hybrid sparse attention architecture designed for long-context language models that improves efficiency through two-level KV sharing between self-decoder and cross-decoder components. It replaces block-level sparsity with token-level sparsity and enables prefill computation to exit early, reducing computational cost and KV-cache storage while maintaining performance on long-context retrieval and multi-turn agent tasks.