Llama.cpp's grammar feature constrains model output to match specified formats by restricting token selection. Using llama-cpp-python, developers can load grammars like json_arr.gbnf to ensure valid JSON responses from models like Llama 2, enabling structured output generation on local devices.
XGrammar-2 is an upgraded framework for structured generation in AI agent applications, introducing Structural Tag—a unified JSON protocol for expressing tool calls, reasoning channels, and custom output formats. It features efficiency optimizations like cross-grammar caching and speculative decoding support, and has been adopted by major AI companies including xAI, Databricks, and DeepSeek.