Hush is a GitHub Action that automatically labels issues using a decision model that returns probability scores, only applying labels when confidence exceeds configurable thresholds. Unlike other LLM-based triage bots, it abstains from labeling when uncertain, costing roughly $0.00002 per issue.
jev-align is an open-source CLI tool from Sutro that helps calibrate AI functions built with TypeSafe's Jev by identifying uncertain predictions, collecting human labels, and using GEPA to iteratively improve model definitions. It supports multiple LLM providers and datasets in CSV, Parquet, and JSONL formats.