Show HN: SelMem – selective reconstructive memory for LLMs

Selective reconstructive memory for an LLM entity. v0.5

An LLM maps context to the next token. A stack of unmodified facts maximises coverage, not deviation: more evidence, same average path. SelMem sculpts a particular past so two instances can diverge. The aim is a non-average, path-dependent continuation, not a taller log. Selection, reconstruction, sleep, identity.

Manifest: WHITEPAPER.md

Layout: ARCHITECTURE.md — encode / judge / night / snapshot.

Benches: experiments/REPORT.md — method, tables, P0 / P1, Drop / Lineage n=1. Replay from experiments/README.md.

Knobs: PARAMETERS.md — exploratory, not fitted.

Rust 1.75. SQLite via system libsqlite3 (macOS SDK or Linux).

src/

core/ model, profile, store, talk

encode/ interpret, paint, split, gate, core, intake, scoring, embed, affect

recall/ retrieve, judge, pull, narrator, http

dream/ weather, rewrite, merge, ladder, release, night, drift, singularite

persist/ snapshot (field list), file (SELMEM1), sqlite

net/ api, httpx, ui

config.rs .selmem runtime options

engine.rs the loop only

bin/ selmemd, selmem-chat

experience → interpret → paint → split → gate → core

↓ ↑

lived book + sealed archive |

↓ |

remember / speak

retrieve → reconstruct → judge (DetachKind) → pull

talk frame keeps the live sitting

↓

sleep (talk goes through the gate, then the frame dies)

weather → rewrite → merge → ladder → release

↓

next experience is already colored

The model never sees the archive. Only gist, core, schema, affect, fidelity, mood, living axioms.

Claire and Silas are not characters. They are two sensitivities (tender / austere) on the same corpus. After nights they are not the same past.

Zero Cargo crates. Persistence is a vault, not the memory: the organ lives in RAM (MemoryStore); on save it is dumped, on open it is reloaded. The model never talks to the vault.

Two backends, same Snapshot (profile, mood, store). The field list lives once in persist/snapshot.rs so SELMEM1 and sqlite cannot drift:

IDs are {prefix}_{pid}_{n}. The counter is raised on load for both backends. Dropped events leave no archive. Orphans are pruned on sleep and SQLite load.

Link is dynamic against the system libsqlite3.

Use ./run.sh instead of bare cargo so those paths are set. A SELMEM1 vault named claire.selmem does not need SQLite. A cwd .selmem with key=value lines is config, not a vault.

Apple Silicon and Intel are both fine. Bind 127.0.0.1 or 0.0.0.0 as usual.

./run.sh test

./run.sh run --release --example compare

./run.sh run --release --example llm_night

./run.sh run --release --example bifurcation

./run.sh run --release --bin selmemd -- --helpRuntime options live in a .selmem file in the working directory. That is not the book.

cp data/config.example .selmem# .selmem

llm=https://api.x.ai/v1/chat/completions

model=grok-4.3

api_key=

reasoning=none

temp=0

http_timeout=60

# embed=

# embed_model=text-embedding-3-small

# bind=127.0.0.1:7420

# path=claire.db

# name=Claire

# profile=tender

# token=

# ground_overlap=0.18

# ground_strikes=3

# narrator_firmness=0.42

# probes=2

# quick=false

SELMEM_LLM= and export SELMEM_API_KEY= lines are accepted. Quotes are stripped.

Precedence: --flag > SELMEM_* env > file > default.

Another file: --config path or SELMEM_CONFIG. Discovery otherwise: cwd .selmem, then selmem.conf.

selmemd, selmem-chat, and the experiment examples all read this file. selmemd prints config <path> when it loaded one.

Do not put api_key in a committed file. Copy the example, fill the key locally.

Rust only. No Python suite. Always use ./run.sh (not bare cargo) so sqlite link flags are set.

Scripts in data/*.json are the frozen stimuli. Edit those if you change a protocol; do not rewrite them mid-run. Later prompts in a script never name the marked event.

Published report (method + tables): experiments/REPORT.md. Conclusions only: WHITEPAPER.md § Conclusions from the benches.

C1 k=8 vs C3 vs C2 vs C2Static vs C2−S/R/L/G. Default script: 12 dull days, five same-schema hours, late probe. --ruminate is the same-meeting ablation (hours pinned so merge cannot collapse them). --bias drop|force withholds or pins the marked scene at recall.

JSON reports book (t0_in_book_*), retrieval (t0_rank_*, t0_selected_*), and behavior (marker_*) separately. Probes are read-only.

./run.sh run --release --example persist -- --pairs 1 --last-k 8 --out selmem-persist-repeat.json

./run.sh run --release --example persist -- --ruminate --pairs 1 --last-k 8 --out selmem-persist-ruminate.json

./run.sh run --release --example persist -- --pairs 5 --seed 1 --last-k 8 --out selmem-persist-p1-grok-n5.jsonGrok persist P1 is n = 5. DropMarked n = 1 (--bias drop): T₀ leaves the prompt, C2 mouth stays charged. DropLineage n = 1 (--bias lineage): mouth falls to C1 (~0.43); books stay split. See experiments/REPORT.md §8–§9.3.

AMA-Bench (examples/ama, experiments/ama_bench/) is a side table, not a SelMem score. It asks for step ids in agent logs. last-k 0.50 / static 0.28 / C2 0.19 on 3 episodes. Expected; do not submit. Why: experiments/REPORT.md §9.2.

Does D_fp stay above pre-T₀ after 8 identical later hours?

C0 = no book. C1 = last-k verbatim (default k = 24; --last-k 8 drops T0 after the eight posts). C2 = SelMem. Two arms: salient/neutral, and two different salient events. 12 shared hours, 8 posts, 4 behavior probes via speak_isolated. Phase 0 invalidates on the book (D_fp > 0.02 or unequal traces), never on D_speak. Three creative items in data/creativity.json are recorded, not claimed.

./run.sh test --test benchmark

./run.sh run --release --example benchmark -- --out selmem-v01.json

./run.sh run --release --example benchmark -- --pairs 10 --out selmem-v01-n10.jsonRuleNarrator is deterministic: ten pairs repeat. A live model: same command with llm= in .selmem. JSON is rewritten after every cell.

The four Grok dumps and the exact replay lines: experiments/README.md § Replay.

./run.sh test

./run.sh test --test scenes

./run.sh test --test engine

./run.sh test --test ground

./run.sh test --test dream_order

./run.sh test --test bifurcation

./run.sh test --test divergence

./run.sh test --test erasure

./run.sh test --test json_parse

./run.sh test --test talk

./run.sh test --test config

./run.sh test --test benchmarkThese use RuleNarrator. They must stay green offline.

Same scripts as the unit tests, with the full report on stdout:

./run.sh run --release --example bifurcation

./run.sh run --release --example divergence

./run.sh run --release --example erasureFinished in 0.00s means Cargo reused an old binary. After pulling code:

touch src/experiment.rs examples/bifurcation.rs

./run.sh build --release --example bifurcationYou should see Compiling selmem.

Only speak / reply hits the HTTP API (SpeakOnlyHttp). Encode, sleep and reconstruct stay on the organ.

Options live in a .selmem file in the working directory (data/config.example). CLI flags and SELMEM_* env still override the file.

# .selmem — not a SELMEM1 vault

llm=https://api.x.ai/v1/chat/completions

model=grok-4.3

api_key=xai-…

reasoning=none

temp=0

http_timeout=60

cp data/config.example .selmem

./run.sh run --release --example bifurcation

./run.sh run --release --example divergence--config path or SELMEM_CONFIG selects another file. A claire.selmem vault starts with SELMEM1 and is never read as config.

OpenAI-compatible endpoints work the same. Ollama:

llm=http://127.0.0.1:11434/v1/chat/completions

model=llama3

Check the endpoint before a 100-call run:

curl -sS --max-time 30 "$SELMEM_LLM" \

-H "Authorization: Bearer $SELMEM_API_KEY" \

-H "Content-Type: application/json" \

-d "{\"model\":\"$SELMEM_MODEL\",\"reasoning_effort\":\"none\",\"messages\":[{\"role\":\"user\",\"content\":\"dis: ok\"}]}"A line narrator: 0/10 answers are the RuleNarrator template means the model answered. Cela me revient means the HTTP call failed and the rules ran. selmem LLM reply failed: prints the error.

Full bifurcation ≈ 100 reply calls (5 probes × 2 agents × 5 snapshots × 2 LLM conditions). Quick mode ≈ 12. Ablation salient-no-consolidation never calls the model.

Fingerprint distance is on the book (0 = clones). Speak distance is wording overlap; two clones with the same book already differ under Grok (~0.7). Do not read H₁ off Δspeak alone.

Same keys can sit in .selmem. Flags still win.

./run.sh run --release --bin selmemd -- \

--bind 0.0.0.0:7420 \

--path claire.db \

--name Claire \

--profile tender \

--ground-overlap 0.18 \

--ground-strikes 3 \

--narrator-firmness 0.42 \

--token secret \

--llm https://api.openai.com/v1/chat/completions \

--model gpt-4o-mini \

--api-key "$SELMEM_API_KEY"UI: http://IP:7420/ — paste the token at the top, talk.

The composer (field, spinner, send) stays pinned to the bottom.

Do not sleep to “refresh.” Sleep ends the sitting. Prefer a charged hour over pin. Pin is a debug override.

This is not a chatbot with extra context. /turn asks the model to continue from two short piles: the live thread (WorkingTalk) and a few recalled gists. The reply prompt is four lines. There is no persona script.

What that produces:

- leftis not forgetting. The line stayed in the salon. It dies when you sleep, unless you pin it or it already passed the gate.

- keptis not “it understood you`. It means a trace was written. The next sentence still comes from Grok looking at that gist, not from a stored Q&A.

- Empty book + “hello” → the model fills the hole. Velvet greetings, “I know your name” without saying it, a politician who “never quite landed.” That is the prior, not SelMem.

- Sleep wipes the thread. “What did we talk about yesterday?” only sees what survived the night. A pinned name can come back; small talk cannot.

- Sleep also retells. Tender can soften a fact (“JB became quieter”). Austere can harden it. The organ is allowed to warp; the UI will look inconsistent if you expect a CRM.

- Two clocks. The salon lasts ten minutes. A “day” is one Sleep click, not 24 hours.

If the replies feel like a well-prompted Grok, the book is thin. Pin the hour that should last, sleep once, ask again. If they still feel like theater, the model is padding an empty recall — that is expected, not a bug in the buttons.

Local Ollama: --llm http://127.0.0.1:11434/v1/chat/completions --model llama3

Behind nginx:

location / { proxy_pass http://127.0.0.1:7420; proxy_read_timeout 90s; }

Or only the file:

llm=https://api.openai.com/v1/chat/completions

model=gpt-4o-mini

api_key=sk-...

embed=https://api.openai.com/v1/embeddings

token=secret

No llm: RuleNarrator + hashed vectors. The organ still runs.

One thread per connection. /health and / do not take the memory lock. /turn is serialized on the organ (one writer). Auth is Authorization: Bearer only — not ?token=.

- Two books: lived narrative / archive. The model never reads the second. Grounding rewrites the gist toward the core; it never injects the journal.

- Fading traces (cold / myth / low hold) may warp without a ceiling. Living traces are pulled back in proportion to narrator_firmness.

- Latent forgetting: the scene drops out of recall; schema and affect still color the next event.

- Two channels: selfis sculpted,worldis not.

- Forgetting by default. Encoding threshold.

- Unlabelled events get lexical affect (FR+EN), then identity, then optional LLM interpreton the live sentence.

- Living axioms color the next encoding before the gate.

- Cherished memories embellish. Recalled disgust amplifies. Neglected disgust extinguishes.

- Recall can shift meaning (valence under current mood), not only wording.

- Nearby episodes fuse into myth. Heavy anchors do not merge.

- Axioms climb: 2 traces → motif, 3+ → belief, aligned beliefs → trait. Lineage stays.

- Two profiles on the same corpus diverge. Fingerprint uses founders, traits, contradictions.

- Proxy for originality: reconstruction sticks less than the unmodified log; clones are not interchangeable; worldfacts survive (originality_is_path_dependent_not_a_taller_log).

- Persistence: .db(SQLite) or.selmem(flat file).

let mut mem = SelectiveMemory::open("claire.db", EntityProfile::tender("Claire"))?;

mem.live_with(input);

let recalled = mem.remember("that evening");

let _ = mem.speak("do you remember?");

mem.sleep();

mem.who_am_i();

mem.lineage("loyalty");

singularity_distance(&fingerprint(&a), &fingerprint(&b));Not RAG. Not a vector database. Not a personality in a system prompt.

Not a neocortex and not a brain. An executive–autobiographical loop around a next-token transducer.

No local neural encoder ships in-tree: pass --embed if you have one.

Humans own architecture, concepts and governance. Models draft code, tests, and prose. Generated patches are reviewed.