Skip to main content
A voice waveform splitting into discrete fact records and a diffuse field of traits

One utterance, two kinds of memory. Facts crystallise; traits diffuse.

Every turn a caller speaks splits two ways. Facts are discrete and retrievable. They are allergic to shellfish. Their manager is Priya. They adopted a beagle in August. Traits are continuous and never stated outright. They get anxious before design reviews. They want the conclusion before the reasoning. Most memory layers store only the first kind. That is why agents that “remember” still sound like they are reading your file back to you. VoiceMemory stores both and injects them differently: facts the agent may repeat, traits it may only be shaped by.

​
What you get

Remembers across sessions

A caller mentions a shellfish allergy on Monday. On Friday they ask about the prawn curry and the agent already knows.

Stays inside the turn

192 ms p50 retrieval, and 93% of that is one OpenAI round trip that prefetching hides.

Three dependencies

livekit-agents, openai, psycopg. No torch, no local models, no multi-gigabyte image.

Multi-tenant by default

tenant_id on every table and every query, with optional PostgreSQL row-level security.

​
How it works

Reading happens inside the voice turn and makes no LLM call. The query is embedded once, classified into one of seven life domains, narrowed to those candidates, ranked in pgvector, and scored. Relevant traits are fetched concurrently. Retrieval also starts speculatively on interim transcripts, while the caller is still speaking, so the network round trip overlaps with speech instead of delaying the reply. Writing happens after the turn and never sits between the caller finishing and the agent speaking. One extraction call pulls out facts, an emotion and traits together. A second call resolves those facts against what is already stored, deciding add, update, delete or nothing.
Scoring is deliberately two-part. base_score is the raw cosine and decides the ordering. Lexical and date bonuses only rescue a buried memory into the result set; they never displace something that was semantically relevant to begin with.

​
Limitations

Stated here rather than buried, because they decide whether this fits your case.
The extraction model reads what was said. There is no prosody analysis, no acoustic emotion, no voiceprint and no speaker identification. Text catches “I’m frustrated”. It cannot catch a flat “fine” said bitterly. Upstream VoiceMem ships those features; this plugin does not, because they require torch, funasr and modelscope.
Injecting memory changes the turn’s chat context, so LiveKit’s equivalence check fails and it cancels the speculative generation it had already started. Preemptive generation is on by default, so unless you turn it off you pay for a discarded LLM call every turn. The plugin logs a warning once at startup if you have not.
“next week” expands to August 31, 2026, matching how the extractor writes dates. Other languages are returned unchanged, so time-scoped retrieval falls back to whatever dates happen to appear in the memory text.
They live in a dedicated voicemem schema so they never collide with yours. The 0.x schema is not stable between minor versions.
on_user_turn_completed does not fire on those turns, so memory is neither injected nor ingested.

​
Start here

Quickstart

Install, migrate, and wire it into an agent. Ends with a transcript proving it remembers across sessions.

Reference

Every Config field, the Python API, the four protocol seams, and the CLI.

Discord

Questions, and what people are building with it.

GitHub

Source, issues, and the changelog.

​
Attribution

VoiceMemory is derived from VoiceMem (Apache-2.0), a separate project by different authors. The prompt templates originate from mem0 (Apache-2.0). See NOTICE and CHANGES-FROM-UPSTREAM.md in the repository. Not affiliated with, endorsed by, or sponsored by LiveKit, the VoiceMem authors, or mem0.