HySparse2 is a hybrid sparse attention architecture designed for long-context language models that improves efficiency through two-level KV sharing between self-decoder and cross-decoder components. It replaces block-level sparsity with token-level sparsity and enables prefill computation to exit early, reducing computational cost and KV-cache storage while maintaining performance on long-context retrieval and multi-turn agent tasks.
Researchers introduce BITCOS, a distribution-adaptive compression layout for ternary LLMs that exploits the actual sparsity of weights (up to 51.5% zeros) to achieve storage below the theoretical 1.585-bit limit, reaching 1.485 bits per weight and improving inference throughput by up to 1.28× on production kernels across multiple platforms.