Sub-200ms local contract guardrails for data pipelines. Airflow, dbt, PySpark, Databricks, Snowpark, Flink, Polars.
That is a real session against samples/deploy.py in this repository — init, fail, fix, pass. Five things are worth noticing:
- initwrites the contract and says what it enabled — two required parameters, 16 secret detectors, six framework profiles auto-detected per file. The rules are visible before the first scan, not buried in defaults.
- deploy_serviceis missing- target_environment. The function runs fine and passes tests; in production it silently takes the dev default. No linter has a rule for this.
- The secret value is redacted in the output — value "s3…" (<redacted>, 20 chars). A tool that reports credentials in CI logs has moved the leak, not closed it.
- fixrewrites it rather than describing it:- DB_PASSWORDnow reads- os.environ["DB_PASSWORD"], with- import osinserted above the existing imports. 18 changes land automatically; the hardcoded default on line 90 is deliberately left as a- manual:item, because- os.environ[...]in a default binds once at import time rather than per call. Thread that one line yourself and the re-run reports- PASS.
- The run states its own scope. 10 in scope, 0 framework-exempt, 2 user-exempt, 1 out of scopeis the denominator behind the result — without it, a repository where every function is framework-owned looks identical to one that complies.
No account, no clone, no config. The analysis path makes no network call.
# 1. Get the binary — no toolchain, ~2MB, no runtime dependencies
# (pick your platform's archive from the Releases page)
curl -sSL https://github.com/kovallent/kv-cli/releases/latest/download/kv-cli-x86_64-unknown-linux-gnu.tar.gz | tar xz
# 2. Point it at a pipeline repository
./kv-cli audit ~/your-pipeline-repoTypical run: under 200ms on a 47-file repository. Exit 0 if compliant, 1 if there are findings.
samples/ ships pipelines that trip every rule, so you can read real output before trusting it on your own code:
git clone https://github.com/kovallent/kv-cli && cd kv-cli
./kv-cli init # created wrote ./.kovallent.yaml
./kv-cli audit --strict samples/ # FAIL 13 errors, 6 warnings — by design
./kv-cli fix samples/deploy.py # FIXED applied 18 changes across 1 file
./kv-cli audit samples/compliant.py # PASS — must always be cleansamples/deploy.py exercises every detector and every suppression path. samples/frameworks/ has one file per supported stack, each demonstrating both the signature exemption and that framework's own detectors.
If you build from source instead of downloading the binary, budget a few minutes for the first compile —
cargobuilds the tree-sitter grammar and needs a C compiler.make demobuilds and then auditssamples/in one step.
Three common causes, all of them by design:
- Your models are SQL. Analysis is Python only. A hardcoded relation in a .sqlfile is not seen.
- No contract file, so defaults applied. Run kv-cli initand enable the rules you want.
- The framework owns your signatures. Check the scope line in the output — if in scopeis low relative to the total, that is the exemption working, not a miss. See Framework support.
kv-cli enforces parameter contracts across Python data pipelines: every governed function must declare the required schema arguments (e.g. target_environment), no credential may be hardcoded, and no infrastructure identifier may be pinned to one environment.
It parses your code locally with a tree-sitter AST — nothing is transmitted, no account is required, and there is no network call in the analysis path — to detect parameter contract breaks (KV001), hardcoded credentials (KV002), and pinned infrastructure identifiers (KV003) before bad code leaves the developer workstation.
The class of bug it targets is the one linters miss: code that is syntactically fine, passes tests, and only breaks when it runs somewhere other than the author's laptop.
Prebuilt binary — no toolchain required. Download the archive for your platform from Releases, then:
chmod +x kv-cli && sudo mv kv-cli /usr/local/bin/
kv-cli --versionEach release ships SHA256SUMS for verification.
With Cargo:
cargo install --git https://github.com/kovallent/kv-cliFrom source:
git clone https://github.com/kovallent/kv-cli && cd kv-cli
make release # -> target/release/kv-cliBuilding from source needs a C compiler for the tree-sitter grammar — Xcode Command Line Tools on macOS,
build-essentialor equivalent on Debian/Ubuntu, and in CI. This applies tocargo installandmake release, not to the prebuilt binary, which has no runtime dependencies.
There is currently no PyPI or Homebrew distribution. pip install kv-cli will not work.
audit is the CI gate. Exit codes are a documented interface — each escalates to different people:
Resolution is first-match-wins in the order 2, 3, 1, 0. An unverifiable run never passes, and drift outranks findings — findings computed against the wrong contract are not a meaningful result.
Add kv-cli to your .pre-commit-config.yaml:
repos:
- repo: https://github.com/kovallent/kv-cli
rev: v0.4.1
hooks:
- id: kv-cliThe hook is declared with language: rust, so pre-commit builds the crate once in an isolated environment on first run. Each contributor therefore needs Rust and a C compiler, and waits through one compile. If that is not acceptable for your team, call the prebuilt binary directly instead:
repos:
- repo: local
hooks:
- id: kv-cli
name: kv-cli contract audit
entry: kv-cli audit
language: system
types_or: [python, yaml]A hook is advisory by construction — --no-verify skips it, and a fresh clone installs none. It is where a finding is cheapest to fix, not where compliance is guaranteed. That is what the CI gate is for.
- KV001— Parameter contract violation (error). A governed function is missing a contract parameter.
- KV002— Hardcoded secret (error). A credential or API key written as a literal.
- KV003— Pinned infrastructure identifier (warning by default). An infrastructure identifier pinned to one environment: a warehouse, catalog, cluster ID, bucket, or endpoint. Warning by default so adopting it does not break CI on day one;- fixnever rewrites these, because the right replacement is a deployment decision.
fix inserts missing parameters into the signature (before **kwargs, preserving multi-line layout) and rewrites hardcoded secrets to os.environ["NAME"], adding import os when needed. audit and fix share one list of flagged bindings, so they can never disagree about what is a secret. Backups go to .kvbak.
Suppress a single line with a trailing # kovallent:allow-secret, or # kovallent:allow-infra for KV003.
Run kv-cli frameworks for the live list. Each profile is scoped to files that actually use the framework — detected from imports, or for dbt from the model(dbt, session) signature. That scoping is what makes aggressive rules safe: account and warehouse would be noisy globally, but in a file that imports snowflake they are precise.
Signatures a framework owns are exempt from KV001. A @dlt.table function takes no arguments because DLT calls it; def model(dbt, session) is fixed by dbt. Injecting target_environment there would break the pipeline, so the exemption overrides the user's own name patterns — a function called daily_sales_job matches *_job, but under @dlt.table it is left alone. Airflow's @dag is owned for the same reason: its parameters become DAG params shown in the UI and in {{ params.* }}, so adding them silently changes the DAG.
User exempt_name_patterns still win over framework rules, so @task def _internal_step() stays out of scope.
A profile can also work the other way: governed_decorators puts functions into KV001 scope. Airflow's @task is the one case — TaskFlow tasks are called from your own DAG body, so the contract parameter can be threaded through at the call site. Declaring it on the profile means the behaviour no longer depends on your contract happening to list a decorator named task.
Governed decorators can be report-only (governed_auto_fix: false), which Airflow's @task is. audit flags a missing parameter, but fix refuses to insert it and prints a manual: item instead — extract_orders("s3://...") would not pass the new argument, so an inserted default would read "dev" in every environment while looking correct. An Airflow-heavy repository therefore cannot reach a clean audit from fix alone; you have to thread the parameter through the call site yourself. This is scoped to the framework: ordinary functions in the same DAG file are still fixed normally.
Select profiles explicitly in the contract if you prefer:
frameworks:
enable: [auto] # or [snowpark, dbt] to apply everywhere
disable: [databricks] # never apply, even when detectedprofiles.yml, dbt_project.yml and any other scanned YAML are checked for plaintext credentials only. {{ env_var('DBT_PASSWORD') }} and ${VAR} are recognised as correct and never flagged. KV003 is deliberately not applied to YAML: a per-target config file is exactly where environment-specific values belong.
- Python: 3.9, 3.10, 3.11, 3.12+
- dbt: def model(dbt, session)model signatures;profiles.ymlcredential scanning.
- Apache Airflow: @dag,@task_group,@setup,@teardown,@taskgovernance; Fernet keys;conn_id, pool, queue.
- PySpark / Databricks: @dlt.table,@dlt.view; PATs; workspace URLs, cluster IDs, DBFS paths.
- Snowpark: @sproc,@udf; session config account, warehouse, role.
- Flink: @udf,@udtf,@udaf; broker and JDBC endpoints viapyflink.
- Polars: storage_optionscredentials and object-store path identifiers.
.kovallent.yaml drives everything: which files are scanned, which parameters are required (with the annotation and default fix writes), which functions are governed (by name pattern, decorator, or all_functions), and the secret detectors. Run kv-cli init for a fully commented template.
version: "1"
rules:
KV001:
enabled: true
severity: error
strict_parameters: true
KV002:
enabled: true
severity: critical
ignore_paths:
- "tests/fixtures/*"The contract stays local. audit --format json emits a semantic fingerprint of it under run.contract_sha256, which a server compares against what is deployed for the repository; drift is reported as its own outcome rather than as a code failure. This keeps audit working offline, on the free local tier, and in air-gapped deployments — none of which survive a mandatory fetch.
The hash covers the parsed contract, not the file bytes, so reformatting or editing a comment is not drift while editing a rule is. run.contract_path is null when the run used built-in defaults.
No service is required. The expected hash has to live somewhere the developer opening the pull request cannot edit — an Actions organization variable is exactly that:
- name: Kovallent gate
run: kv-cli audit --strict --format json --expect-contract ${{ vars.KOVALLENT_CONTRACT_SHA }}On mismatch the run exits 3, distinct from findings (1) and tool errors (2).
Committing the expected hash to the repository instead is weaker — whoever weakens the contract can update the lock in the same commit.
Known gap: anyone with write access can edit the workflow to drop the flag. Close it with CODEOWNERS on .github/workflows/ plus branch protection, or an organization ruleset that requires the check. See ADR 0001 for rationale, rejected alternatives, and staging plans.
schema/findings.v1.json is generated from the Rust types by kv-cli schema; a test asserts the committed document matches and that a real run validates against it, so the two cannot drift.
- Version and identity. runcarriesschema_version(bumped only on a breaking payload change, independent oftool_version), run identity (repo,commit,branch,timestamp), contract provenance, and a scope block. Identity is never guessed:repoandcommitcome from the CI environment (GITHUB_REPOSITORY,CI_COMMIT_SHA, …) or from--repo/--commit/--branch, andidentity_sourcerecords which —"flag","env:GITHUB_SHA", or absent.
- Severity reporting. Severity is reported as emitted. errorsandwarningscount intrinsic severities regardless of--strict; the flag is emitted so a consumer can apply the policy itself. Otherwise a strict run reports every finding as an error and the payload cannot say how many were warnings.
- Completeness. No file can vanish from the payload. Every file kv-cliresolves for scanning is either represented infindings/files_scannedor named inrun.skipped[]as{path, reason, severity, detail}, reason being"syntax_error"or"unreadable". A syntax error is a warning — matching howKV003was introduced, it does not fail CI by default and is promoted under--strict. An unreadable file is always an error, since there is no lenient reading of "the tool could not check this file."
- Scope block. The scope block is the evidence behind a green result. Without it, a repository where every governed function is framework-owned looks identical to one that fully complies:
functions: 10 in scope, 0 framework-exempt, 2 user-exempt, 1 out of scope
Framework ownership and the contract's exempt_name_patterns are counted separately — the first is ours, the second is the customer's choice.
src/python.rs parses each Python file with tree-sitter (tree-sitter-python) and reads everything off the parse tree. Nothing in the Python analysis is text matching.
That matters most for fix, which rewrites source. The parser yields string literals already paired with what binds them, so a, b = "x", "y" pairs element-wise. An earlier text-scanning implementation walked backwards through bytes to guess the binding, and on that line it paired b with "x" — then rewrote the wrong literal, breaking a and leaving the secret in place. Bindings are recognised in five forms: assignment (including annotated and tuple targets), dict entries, keyword arguments, walrus expressions, and parameter defaults.
Signatures come from the tree too, so multi-line parameter lists, async def, lambda defaults, / and * separators, and decorator calls spread across lines all work with no special cases.
- A parameter default (def f(password="...")) is reported but never auto-fixed:os.environ[...]in a default is evaluated once at import time, not per call, so the rewrite would change behaviour.fixprints it as amanual:item.
- A file with syntax errors is audited on tree-sitter's partial parse and flagged in the output, but fixrefuses to rewrite it.
YAML is handled separately by src/yamlscan.rs, a line scanner rather than a parser: serde_yaml would give the structure but not line numbers, and a finding without a line number is not actionable. It tracks block scalars so prose inside a | block is not mistaken for mapping keys.
detect-secrets finds high-entropy strings; KV002 finds a literal assigned to a name your contract forbids, which also catches password = "dev". Neither has any concept of "this function must accept target_environment."
make test # unit tests (121)
make check # fmt --check + clippy -D warnings + tests
make demo # build, then audit samples/ (exits 1 by design)samples/compliant.py must always audit clean. cargo run --example dump <file.py> prints the parse tree with field names — useful when adding a rule that needs a node kind you have not handled yet.
- Build dependencies. Building requires a C compiler (cc) for the tree-sitter grammar. The output is still a single binary with no runtime dependencies.
- serde_yaml. Pinned at its final 0.9 release and no longer maintained upstream. Swapping it for serde_norwayorserde_ymlis a drop-in change; the contract fingerprint depends on it, sogolden_fingerprintmust stay green through the swap.
Contributions are welcome — a new AST rule, a parsing improvement, a bug fix.
- Fork and branch (git checkout -b feature/new-rule).
- Open a thread in Discussions if you want to talk it through first.
- Run make checkand open a pull request.
Released under the MIT License.
- Announcements — release notes and roadmap.
- Ideas and feature requests — propose a KVrule or a framework profile.
- Q&A — setup, AST rules, custom configuration.
- Kovallent Enterprise waitlist — team-wide policy governance and scorecards.