Write Linters and Tools Before Code

Write Linters and Tools Before Code

Why linters and tools must exist before non-deterministic agents do the gruntwork.

Overview

A language model samples tokens. Two runs of the same prompt can produce two different documents, and the difference is rarely just wording. It shows up in structure: a skipped heading level, a code fence without a language, a curly quote where a straight one belongs, frontmatter fields in the wrong order. We keep treating that variance as a prompt engineering problem. It is not. It is the operating condition, and the workflows that hold up are the ones built around it.

The productive move is to stop asking a model to invent structure and to start giving it a structure to fill in. That structure has to be executable: a parser, a linter, a schema validator, a builder. Write those first. Then let the agents do the gruntwork.

Non-Deterministic Agents

An LLM is a very good local writer and a very bad global bookkeeper. It can produce a correct paragraph on demand. It cannot reliably hold a document invariant across a thousand lines, because it has no persistent program state and no notion of "the rest of the file". Its only memory is the context window, and attention over that window is uneven: instructions near the beginning and the end dominate, while the middle decays.

Reducing the temperature reduces variance, but it never removes it. The same model, given the

same prompt, will still sometimes choose an unordered list where you expected a paragraph, or

a

####

headline where your renderer expected a

###

.

None of these are hallucinations in the usual sense. They are valid continuations of an underconstrained problem. The model is answering the question you actually asked, which is "produce something plausible", not "produce this exact structure".

That gap is where generated documents rot. The content is fine; the container is wrong. You do not fix it by asking or prompting harder. You fix it by making the container machine-checkable and making it fail until the expected outcome is verified.

Linters as Executable Architecture

A style guide is a suggestion. A linter is a constraint. Architecture that only exists in prose is architecture a model can drift away from, because prose in a prompt is context, and context is advice. A linter is different in kind because it parses the artifact, applies a ruleset, prints errors with line numbers, and exits non-zero.

That is the whole value proposition. The linter turns "is this document good?" into a deterministic oracle. It is only possible to pass or fail, plus it outputs a list of precise reasons. Non-deterministic systems can be optimized against deterministic oracles. They cannot be optimized against tastes or styles.

For any target format you should encode, at minimum:

- which elements are allowed, and at which nesting depth

- which tokens are legal, and which must be normalized

- which character set is required

- which sections are mandatory, and in what order

- which metadata fields are required, and how they are typed

Every rule you write is a decision the model no longer has to make in the middle of a generation.

Decisions Versus Gruntwork

Think of a generation as a walk through a tree of decisions. Each decision is a branch, and a non-deterministic system can take a wrong branch at any of them. Error probability compounds with decision count, which is why long documents degrade. There are simply more decision branches, and the model re-derives each one from context instead of following a fixed plan.

The fix is to move decisions out of the model and into the tools. A linter captures structural decisions. A schema captures data decisions. A formatter captures presentation decisions. A builder captures byte-level decisions.

What is left for the model is only the gruntwork. Describing a concept, summarizing a source, filling in a section, translating a specification into prose.

Gruntwork is where agents excel at. It is high-volume, low-branching, and verifiable.

The more architecture you encode in linters and tools, the more of the job becomes gruntwork, and the better the outcome will be. That is the entire thesis in one sentence.

The Markdown Format Case

Markdown is the worst case, because it is permissive by design. There is no single grammar, every renderer implements a dialect, and almost any byte sequence parses as something. Agents exploit that ambiguity without meaning to.

They emit an

h1

where the site expects a single

h2

title. They jump from

h2

to

h4

.

They open a fenced block with no language, or draw tables with Unicode box characters instead

of using an actual markdown table. They produce typographic quotes and dashes that a strict

pipeline will not accept. They omit required sections and frontmatter entirely.

None of that is visible until something downstream breaks. A strong linter makes it visible immediately, and a good linter error is a better prompt than any style guide, because it names the exact line and the exact rule.

For example, the toolchain behind this website enforces a small but non-trivial ruleset for every article:

- exactly one h2, and it must match the title

- h3and- h4headlines, ASCII only, shorter than forty characters

- no h5, and noh4without a precedingh3

- fenced code blocks with a known language, and no box characters inside them

- body text restricted to a mapped ASCII character set

- dictionary terms written as [Abbr]{Long Form}so they render as abbreviations

- a mandatory ### Overviewand### Referencessection

The article you are reading is written under those rules and passes that linter. That is not a detail of taste and style, but the encoding of architecture and structure into the document format.

The constraints shaped what could be produced, and the result is more consistent than free-form generation otherwise would have been.

The linter is a single command, and can be used per-document or per-folder, depending on whether the LLM agent needs to generate a single document or a batch of documents:

go run toolchain/lint.go ../weblog/public/weblog/articles/write-linters-and-tools-before-code.md; go run toolchain/lint.go ../weblog/public/wiki/articles/first-chapter/*.md;

It exits zero, or it prints every violation with a line number and exits one. There is no middle ground, and no room for a model to negotiate the ruleset.

Binary Formats Need Schemas

Markdown is forgiving. Office formats are not.

A

docx

,

xlsx

, or

pptx

file is actually a

zip

file holding an Open Packaging Conventions

tree of

ooxml

parts, each validated against an

XSD

schema. A

pdf

is a graph of numbered objects with a cross-reference table. A

csv

has a dialect

and a column contract. A

json

or

yaml

document has a schema you hopefully wrote down in

a parser or linter.

If you let a model emit the raw bytes, you get artifacts that are well-formed enough to look statistically plausible and invalid enough to fail the moment a real application tries to open them. The first line of defense is a schema validator, paired with round-trip checks.

- reopen the generated file

- revalidate the structure that matters

- rebuild the output format

There is a second tool that matters more than the validator, the parser and the builder.

Do not let the model write bytes at all, and enforce the use of the same parser that's used for rendering and for linting.

Let the LLM agent emit structured data via API calls, and let a deterministic tool turn that data into the final artifact. The model chooses values and the tool serializes. This removes an entire category of failure, because serialization is exactly the kind of task a non-deterministic system is worst at and a library is best at.

The same discipline applies to the bytes themselves. Every one of these layouting formats is messy by nature. Elements may appear in any order, namespaces and prefixes are optional, whitespace is interchangeable, and the same document can be serialized in dozens of equivalent ways. A schema alone cannot capture that, because the mess lives in the layout, not in the vocabulary.

What you need is a lexer and a parser that turn the bytes into a syntax tree, plus a linter

that walks the same tree with an advanced ruleset. The tool that reads the format becomes the

tool that judges it. For

ooxml

I built exactly that kind of linter on top of

godocx

, reusing

its parser rather than writing a second one.

The ruleset is what forces a messy layout back into an expected structure.

A Practical Workflow

The order of operations matters more than any individual tool:

- Specify the format as a grammar, a schema, and a set of invariants.

- Write a parser and a linter that report line-numbered errors and exit non-zero.

- Add deterministic fixers for everything that can be normalized automatically.

- Write builders for binary formats so the model composes calls instead of bytes.

- Let the agent do the gruntwork, then lint and fix in a loop until zero errors.

- Gate the repository on the linter so the contract cannot silently weaken.

Steps one through four are architecture work. Steps five and six are the payoff. It feels slower to build the gates first, and it is, right up until the first time an agent produces two hundred documents overnight and every one of them is structurally valid.

The loop is boring on purpose:

while true; do

agent write --out article.md;

go run toolchain/lint.go article.md && break;

agent revise article.md < lint.log;

done;

Nothing in that loop relies on the model remembering a rule. The rule lives in the linter, and the linter is the only authority that matters.

Linters and tools are how you convert taste into a specification and a specification into a judge. For deterministic software that is a nice property. For non-deterministic agents it is the only thing that scales, because the model's variance never goes away, it only moves along based on context. Encode the architecture, and the variance lands where it can be caught.