Credence is an embeddable local inference runtime for typed probabilistic decisions from GGUF language models. It is built on a pinned llama.cpp dependency and is intentionally independent of any agent framework.
A decision is scored from the model's next-token distribution over a fixed set
of permitted labels. No text is generated, so a call costs one prompt decode
and returns a boolean, the probability the model assigned to true, and the
uncertainty diagnostics needed to know whether that probability is worth
anything.
Phase 1 is in progress. The repository has a real model-backed boolean decision path, explicit profile-bound labels with stated semantics, a Qwen3 ChatML-style profile, a generic raw-text profile that resolves labels on BPE vocabularies, single- and multi-token scoring, top-token diagnostics, two labelled evaluation sets, and Platt (temperature plus bias) calibration fitted from them. The public C ABI, multi-choice operations, daemon, and MCP adapter are later phases; see the roadmap below.
The canonical local configurations are CMake presets:
cmake --preset mac-cpu-golden
cmake --build --preset mac-cpu-golden
ctest --preset mac-cpu-goldenmac-metal-release is the fast configuration on Apple Silicon. Every build
preset has a matching test preset. The test suite includes an end-to-end CLI
check that is skipped, not failed, until the smoke model below is fetched.
Model files are never committed. tools/fetch-test-models.py owns the pinned
manifest and verifies SHA-256 on every fetch:
python3 tools/fetch-test-models.py # smoke model
python3 tools/fetch-test-models.py --name qwen3-4b # demo modelThe 0.6B model exercises every code path but is too small to make reliable
decisions: on the checked-in 22-case evaluation it answers true for almost
everything. The 4B model gets 21 of 22 raw and 22 of 22 after calibration.
See docs/phase-1-progress.md for the numbers.
Probe a model:
build/mac-cpu-golden/credence probe --model tests/data/models/Qwen3-4B-Q4_K_M.ggufMake a boolean decision:
build/mac-cpu-golden/credence decide \
--model tests/data/models/Qwen3-4B-Q4_K_M.gguf \
--profile profiles/qwen3/qwen3-decision-v2.json \
--context "Invoice #4471 was issued on 3 March. The full balance remains unpaid." \
--proposition "The invoice has been paid."The result is JSON with value, raw_probability (always the probability of
true), calibrated_probability (null until a calibration is supplied),
entropy, top_two_margin, answer_conformity, the competing top_tokens,
and the model and profile identifiers the numbers belong to. Pass
--untrusted-evidence to wrap the context in the profile's untrusted
delimiters; the result reports evidence_marked_untrusted so callers can see
that the flag took effect.
Without --profile, the built-in generic profile renders a plain instruction
prompt ending in Answer: and resolves A/B labels, falling back to a
leading-space encoding when the vocabulary merges the label into the prefix.
tools/evaluate.py runs a labelled JSONL set through the CLI and reports
accuracy, mean negative log-likelihood, Brier score, and the minimum answer
conformity, so a formatting failure is visible separately from a decision
failure:
python3 tools/evaluate.py \
--credence build/mac-cpu-golden/credence \
--model tests/data/models/Qwen3-4B-Q4_K_M.gguf \
--profile profiles/qwen3/qwen3-decision-v2.json \
--evaluation tests/data/evaluation/boolean-smoke.jsonl \
tests/data/evaluation/boolean-business.jsonl \
--verbosetools/calibrate.py fits a calibration profile on the same data:
python3 tools/calibrate.py \
--credence build/mac-cpu-golden/credence \
--model tests/data/models/Qwen3-4B-Q4_K_M.gguf \
--profile profiles/qwen3/qwen3-decision-v2.json \
--evaluation tests/data/evaluation/boolean-smoke.jsonl \
tests/data/evaluation/boolean-business.jsonl \
--output calibration.jsonThe profile is Platt scaling, sigmoid(logit(p) / temperature + bias), fitted
with Platt's smoothed targets so a set the model already separates still has
a finite optimum. Pass --calibration calibration.json to credence decide;
value then follows the calibrated probability and the output says so in
decision_basis. The fitter warns when the set is small. The checked-in sets
are 22 cases: enough to demonstrate the mechanism, not enough to trust the
resulting numbers in production.
See docs/probability-semantics.md for what
each probability field does and does not mean.
Credence ships as a self-contained native package per operating system rather than a single universal format:
./build.shOn Apple Silicon this builds the Metal package; on Intel macOS it selects the
CPU-golden preset. The archive is written to dist/ as
credence-v<version>-macos-<arch>.tar.gz.
On Windows, run build.bat for the CPU package, or set
CREDENCE_WINDOWS_VARIANT=cuda before running it for the CUDA package. The
result is dist/credence-v<version>-windows-x64-<cpu|cuda>.zip.
- Boolean decisions, profiles, scoring diagnostics, calibration (current).
- choose,- score, and- rankoperations over explicit candidate sets.
- Model and profile fingerprints enforced on every calibration and result.
- Stable C ABI and libcredencefor embedding.
- Resident daemon so the model is loaded once, plus an MCP adapter.
- Language bindings and family profiles for Llama and Gemma models.
Copyright 2026 Sonus Immersivus. Credence is released under the
Apache License 2.0. You are free to use, modify, and redistribute
it, commercially or otherwise, as long as you keep the copyright notice and
the attribution in NOTICE. llama.cpp is bundled under its own MIT
license; see THIRD_PARTY_NOTICES.md.