Before fine-tuning Laya, try specializing its representation space.
Laya Studio helps you specialize a frozen Laya model for a new domain or decision task—without gradients, without changing its weights, and without maintaining another full model checkpoint.
Describe the behavior you want in natural language. Laya Studio generates synthetic examples, derives lightweight specialization geometry from Laya's existing representations, benchmarks multiple approaches against vanilla Laya on independent test data, and exports the best result as a small portable artifact.
The intuition is simple: the model may already contain enough information for your task. Sometimes you do not need to teach it again—you need a better way to read the representation it already learned.
Suppose vanilla Laya needs to classify support tickets as CRITICAL, NORMAL, or IGNORE.
Laya Studio can:
- turn that requirement into an explicit, editable decision policy;
- generate diverse labeled examples for specialization;
- derive centroids, contrastive directions, transforms, or steering vectors from those examples;
- test every method against vanilla Laya on independently generated held-out cases;
- let you inspect edge cases and paraphrases in the Playground;
- export the winning specialization and reload it in another application.
Traditional fine-tuning
task → dataset → gradient training → another model checkpoint
Laya Studio
task → synthetic examples → representation specialization → small portable artifact
The base checkpoint remains frozen and reusable. One Laya installation can support many independent specializations without duplicating its weights.
Our experiments show that changing how Laya's embedding or representation space is interpreted can produce substantial improvements on some classification, routing, prioritization, ranking, and domain-decision tasks. It does not work universally—and Laya Studio is deliberately built to tell you when it does not—but it works often enough that it is worth testing before training new weights.
If representation-space specialization reaches the quality and robustness your task needs, fine-tuning becomes unnecessary for that task. If it does not, the experiment gives you evidence that training is justified rather than assumed.
Practical advantages include fast iteration, tiny artifacts relative to full checkpoints, simple rollback, direct comparison with baseline Laya, and a much lower barrier to trying several task definitions on a laptop. No optimizer or backward pass is used, and the original model remains available for every other specialization.
If you find the idea useful, consider starring the repository—it helps more people discover and test the approach.
Laya's encoder may already organize task-relevant information in its latent space. Laya Studio tests whether simple, inspectable geometry can exploit that organization:
- Prompt-only: improve the task formulation without changing representation geometry.
- Prototypes and centroids: classify by similarity to labeled regions of Laya's own embedding space, using means, medoids, nearest examples, or top-k voting.
- Contrastive vectors: derive directions such as mean(positive) - mean(negative).
- Whitening and residual transforms: center, whiten, or analytically transform embeddings with explicit, exportable statistics and small matrices.
- Multi-vector methods: represent a task through several semantic directions instead of one global axis.
- Pairwise ranking: decompose ranking decisions and aggregate deterministic comparisons.
- Activation steering: where the backend exposes a safe hook, inject a derived task vector into an intermediate representation before Laya's original decision head.
These methods are evaluated against the unchanged Laya baseline on exactly the same hidden data. The framework reports accuracy, balanced accuracy, macro and per-class F1, precision, recall, Brier score, ECE, log loss, latency, throughput, specialization cost, and peak memory. Robustness checks cover option order, paraphrases, generator style, irrelevant noise, and domain holdouts.
Repeated seeds produce paired baseline differences, mean, standard deviation, 95% confidence intervals, seeded bootstrap intervals, and task-level win/tie/loss counts. A method is not presented as better because it happened to win one benchmark.
This remains an experiment harness, not a claim that representation specialization always works. Negative, unsupported, and statistically unstable results are retained alongside positive ones.
python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'For exact known-good dependency versions, add -c requirements/constraints.txt to the install.
PyTorch Laya:
pip install -e '.[pytorch]'
export LAYA_LAB_BACKEND=pytorch
export LAYA_LAB_MODEL=convaiinnovations/layaApple Silicon (macOS 14+, Python 3.11+):
pip install -e '.[mlx]'
export LAYA_LAB_BACKEND=mlx
export LAYA_LAB_MODEL=aac6fef/laya-mlxThe current MLX public API does not expose a safe intermediate-state hook, so
activation_steering is recorded as unsupported there. Embedding strategies still reuse the
loaded MLX encoder. PyTorch freezes every parameter and provides a temporary encoder-layer hook
for true activation-steering experiments.
Copy .env.example to .env, export its values in your shell, or save the connection from the
Studio. .env is ignored by Git. API keys are never stored in suites, experiment metadata, logs,
or browser local storage.
laya-studio serveOpen http://127.0.0.1:8787, then:
- describe the desired behavior in natural language;
- connect OpenRouter, OpenAI, or another OpenAI-compatible endpoint;
- review and edit the inferred policy and labels;
- generate isolated specialization, validation, hidden, paraphrase, and hard datasets;
- review individual examples without handling raw JSON;
- select the methods to compare on real frozen Laya;
- keep a default method, test modified examples in the Playground, and export the result.
The Model library gives each specialization a readable name and lets you keep, remove, compare, or export individual methods. The separate Playground can load benchmark examples, alter their wording without saving those edits, and run them through any retained method.
By default an API key remains only in process memory. The optional Save this connection to
.env control writes the local Git-ignored file with restrictive permissions and never sends the
saved secret back to the browser. A separate provider or model can generate the benchmark.
laya-studio specialize \
--task "Prioritize technical support tickets" \
--labels URGENT,NORMAL,IGNORE \
--examples 50 --method multi_vector_steering \
--output exports/support-priority-layaFor a reviewed policy, use --task-file task.yaml. The command obtains validation examples from
the independent benchmark provider, fits without gradients, exports, starts a fresh Python process,
reloads deterministic probes, and rejects the export if predictions or probabilities drift.
Model A can generate specialization data while model B independently generates validation and hidden data:
export LAYA_LAB_LLM_API_KEY=...
export LAYA_LAB_SPECIALIZATION_MODEL=model-a
export LAYA_LAB_BENCHMARK_MODEL=model-b
laya-studio generate \
--domains 10 --tasks-per-domain 5 \
--specialization-examples 40 --test-examples 300 \
--style-control-model model-c \
--repetitions 3 --seed 100 --output benchmarks/research
laya-studio benchmark \
--suite benchmarks/research/seed-100 \
--methods baseline,prompt_only,nearest_prototype,contrastive_vector,multiclass_centroids,whitened_prototypes,residual_embedding_transform,activation_steering,multi_vector_steering,pairwise_ranking \
--seeds 11,22,33--style-control-model adds an independently prompted hidden stratum. Unsupported method/task
combinations produce explicit skip artifacts. On PyTorch, names such as activation_steering@0,
activation_steering@11, and activation_steering@-1 compare encoder layers; strength is always
selected on validation data only.
Inspect paired statistics and held-out domains with:
laya-studio report RUN_ID --bootstrap-samples 10000
laya-studio report RUN_ID --holdout-domain smart_home
laya-studio compare RUN_A RUN_Bflowchart TD
T[Task specification] --> SG[Specialization generator: model A]
T --> BG[Benchmark generator: model B]
SG --> S[(Synthetic specialization examples)]
BG --> V[(Validation)]
BG --> H[(Hidden / paraphrase / hard tests)]
S --> M[Closed-form strategy fit]
M --> L[Frozen Laya + specialization artifact]
V --> L
L --> E[Evaluation]
H --> E
E --> R[(SQLite, artifacts, report, dashboard)]
L --> X[Portable export + fresh-process fidelity gate]
The benchmark prompt never receives specialization examples. Validation may select a scalar
strength, but hidden, paraphrase, and hard examples never reach fit. Every row receives a hash of
normalized text, and suite loading fails if the same content appears across protected splits.
The specialization and benchmark generators may use entirely different providers and models. If they share a provider, their prompts and seed ranges remain isolated. Every raw response is cached, so an experiment can be replayed without contacting the provider again. All generated output is validated through strict Pydantic schemas and is treated as data, never executable code.
See the technical design for inspected Laya APIs and backend boundaries.
Prototype classification is not described as internal Laya steering. It is interesting for a different reason: it asks whether the encoder already exposes enough task structure to make a new decision without retraining the model. Centroids, contrastive directions, whitening, residual transforms, and multi-vector strategies are lightweight ways to test that geometry directly.
Residual transforms are explicit small matrices derived by ridge linear algebra. No strategy calls
an optimizer or backward, and all Laya parameters have requires_grad=False under PyTorch. True
activation steering is reported separately because it modifies an intermediate representation and
then uses Laya's original decision head.
A lightweight export references the base model and stores only specialization state:
support-priority-laya/
├── manifest.json
├── specialization.json
├── vectors.safetensors
├── processor.json
├── fidelity.json
└── README.md
from laya_studio import SpecializedLaya
agent = SpecializedLaya.from_pretrained("./exports/support-priority-laya")
result = agent.predict("Production is down for every customer")
# agent.system_one(...) preserves the same entry pointExport a retained specialization:
laya-studio export 'RUN_ID:TASK_NAME:STRATEGY' --output exports/my-laya
laya-studio export 'RUN_ID:TASK_NAME:STRATEGY' --output exports/portable --self-containedThe default bundle does not duplicate base weights. --self-contained accepts a resolved local
checkpoint directory and copies it into the artifact; it never silently downloads weights. The
manifest records license and attribution metadata, but redistribution remains subject to the
upstream checkpoint's license.
Every export runs deterministic probes before serialization, reloads the artifact in a fresh Python process, reruns those probes, and rejects the export if labels or probabilities differ beyond tolerance.
Suites are ordinary files and can be rerun without an LLM:
laya-studio benchmark --suite benchmarks/my-suite --backend pytorch --seed 42SQLite records the model identifier and revision when available, backend, task, immutable suite, strategy parameters, metrics, timings, seeds, environment, and Git commit. Adjacent run directories retain specialization matrices and explicit skips. Experiment databases, generated datasets, fitted vectors, and exports live under local Git-ignored directories; users own their results.
To add a strategy, subclass Strategy in src/laya_studio/strategies.py, implement fit, declare
the final decision_component, return JSON metadata plus named arrays, register its CLI name, and
add export/reload fidelity coverage. fit must not accept a hidden split.
To add an LLM provider, implement LLMProvider.generate_json in
src/laya_studio/providers.py. Preserve strict JSON validation, raw-response caching,
provider/model/seed provenance, and the specialization/benchmark boundary. Never execute output.
To add a runtime, implement LayaBackend in src/laya_studio/backends.py. Batch embeddings and
decisions wherever possible. Only advertise activation steering when a stable, inspectable hook
exists before the original decision head.
benchmarks/smoke contains ten human-authored fixture tasks across five domains. It exists for CI
and plumbing, not as scientific evidence. Run it against a real checkpoint with:
pip install -e '.[pytorch]'
laya-studio benchmark --suite benchmarks/smoke --backend pytorch \
--model convaiinnovations/laya --seed 17On Apple Silicon:
pip install -e '.[mlx]'
laya-studio benchmark --suite benchmarks/smoke --backend mlx \
--model aac6fef/laya-mlx --seed 17- Representation specialization does not universally replace fine-tuning. Some required behaviors will not be captured reliably by pooled embeddings, small transforms, or steering vectors.
- Laya's encoder was not trained as a universal bi-encoder; cosine geometry may be weak even when the typed decision head is strong.
- A pooled sentence direction is dimensionally valid at an encoder layer but is not guaranteed to be a causal concept direction.
- Synthetic examples can encode provider bias. Independent generation, style-control strata, domain holdouts, and human review reduce that risk but do not eliminate it.
- Provider seeds may be advisory. Cached artifacts, not regenerated text, are the reproducibility boundary.
- Validation selection can overfit small synthetic sets. Use independent generations, multiple seeds, paired intervals, generator-style strata, and held-out domains.
- Normal confidence intervals are approximate for few tasks; prefer bootstrap output and inspect task-level wins, ties, and losses.
- ru_maxrssis process-level peak memory and can overstate incremental strategy memory.
- Self-contained checkpoint redistribution remains subject to upstream licensing.
Fine-tuning remains appropriate when the behavior you need cannot be represented reliably through these lighter-weight methods. Laya Studio's job is to make that boundary measurable.
pytest
ruff check src tests scriptsFast tests use the fake backend and cover leakage protection, deterministic SafeTensors,
prototype and contrastive mathematics, metrics, option permutations, artifact versioning,
fresh-process export fidelity, backend contracts, LLM-response validation, and malformed suites.
Mark tests requiring downloaded weights with @pytest.mark.integration.