Modal shares optimization techniques for serving large language models powering coding agents at scale, demonstrating how to achieve 2.8x performance improvements per user and 5.6x across users through inference engineering. The article explains the hardware requirements and workload characteristics necessary to economically operate trillion-token inference services for trillion-parameter models like Moonshot's Kimi K2.6.
Anthropic's Claude has optimized over 30 open-source biomolecular modeling tools, achieving roughly 4x average speedup and creating a low-memory mode for larger predictions on single GPU nodes. The company is open-sourcing the optimized code and launching a protein design competition with Adaptyv Bio offering up to $1 million in Claude credits and wet lab validation for over 5,000 designs.