TileRT achieved 469 tokens per second on AMD Instinct MI355X GPUs, topping the InferenceX AgentX leaderboard for inference performance on real-world coding agent workloads. The benchmark measures sustained performance across long-horizon agent tasks with growing context, where TileRT retained approximately two-thirds of its speed even at one-million-token context lengths.
Debian Inference Portal is a self-service platform for Debian contributors to access shared LLM inference through an OpenAI-compatible API, funded by Scaleway's monthly credit sponsorship. Contributors log in via Salsa, manage API keys, and track usage with weekly soft limits to ensure fair resource distribution across the community.
Directional steering is a runtime technique for DS4 that modifies transformer model outputs by applying learned direction vectors to control behavior like verbosity. Users can build custom directions from paired prompt lists and apply them during inference with adjustable scales to amplify or suppress specific model characteristics.