Typed probabilistic decisions from any LLM. No text generation.
jev-serve exposes a /v1/systemone endpoint that scores structured decisions via first-token logit readout — the same approach openjev.com uses in the browser, running server-side on any MLX model or OpenAI-compatible API.
- 34× faster than structured JSON generation (0.23s vs 7.80s per decision)
- Full probability distributions — not a point estimate, a calibrated pper option
- Three question types: choice(pick one),noul(0–1 probability),score(ordinal level)
- Two backends: direct MLX inference or any OpenAI-compatible API (ollama, etc.)
- Apache 2.0 — derived from kev by Jared Palmer
# MLX backend (Apple Silicon)
uv pip install -e ".[mlx]"
# OpenAI-compatible API backend
uv pip install -e ".[api]"# MLX — direct inference on any local model
jev-serve --mlx lmstudio-community/Qwen3.8-27B-MLX-6bit
# API — logit readout via ollama, LM Studio, OpenAI, etc.
jev-serve --api http://localhost:11434/v1 --api-model gemma4:e4b-mlxcurl -X POST http://localhost:8008/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"state": "2010 Infiniti G37 S, 6MT, daily driver, performance-oriented owner.",
"questions": {
"top_mod": {
"type": "choice",
"instructions": "What is the most popular upgrade category?",
"criteria": {
"exhaust": "Exhaust",
"intake": "Intake",
"wheels": "Wheels",
"audio": "Audio"
}
},
"wants_power": {
"type": "noul",
"instructions": "Is this owner primarily interested in power gains?"
}
}
}'{
"answers": {
"top_mod": {
"type": "choice",
"choice": "exhaust",
"confidence": 0.51,
"probabilities": { "exhaust": 0.64, "intake": 0.03, "audio": 0.05, "wheels": 0.28 }
},
"wants_power": { "type": "noul", "noul": 0.87 }
},
"latency_ms": 190
}20 products, same 4-option choice question, sequential, no batching.
Scratch mode runs one forward pass and reads a single logit position. The decoder generates tokens one by one until EOS, then parses JSON. The gap widens with option count and output length.
Instead of generating text, jev-serve reads the model's next-token logit distribution at the boundary position (after the prompt), picks out each option's first token, and normalizes with softmax. One forward pass per question; no sampling, no JSON parsing, no hallucinated fields.
This is identical to what openjev.com does in the browser with wllama.
Portions derived from kev, Copyright 2026 Jared Palmer, Apache 2.0.