During trial for a former Air Canada employee accused of orchestrating Canada's largest gold heist in April 2023, an Air Canada manager testified that security at Toronto Pearson's cargo warehouse was so lax that employees commonly shared login credentials and left computers unattended with critical systems accessible. The $22 million heist involved using a manager's credentials to remove a 'valuable' code from the shipment, causing 400 kilograms of gold to be handled as routine cargo and misplaced in an unsecured warehouse location.
Eval engineering is the discipline of building production-grade evaluation systems that proactively govern AI behavior at scale, ensuring quality standards are enforced in real-time rather than after deployment. Unlike traditional testing, eval engineering treats evaluation as ongoing infrastructure that encodes domain expertise into automated systems to prevent AI errors from reaching users, as illustrated by Air Canada's chatbot that provided incorrect bereavement fare information.