Mnemo: A Portable Long-Term Memory Layer for Local LLMs

Your Ollama model can already answer questions, but every conversation still starts from zero. The project decision you discussed yesterday, the preference you set today, and the constraint you need tomorrow all disappear.

That is the memory gap in local LLM workflows. Ollama gives you a model service that can answer. What it answers, and whether it remembers the constraints you gave it earlier, still depends on how much context you paste into each prompt.

Mnemo is not trying to rebuild RAG. It uses a knowledge graph plus entity extraction to manage long-term memory, so a local LLM can remember project decisions and entity relationships instead of asking again, “What does this API do?“

1. What Mnemo Is: Positioning and Core Capabilities

1.1 Reading the Positioning

The phrase “local-first AI memory layer” has two important parts:

• local-first: the data stays on your machine, is portable, and does not depend on whether a SaaS product survives
• memory layer: it is not another RAG framework or another agent framework. It focuses on memory itself: entity extraction, graph construction, and semantic retrieval

For example, when you ask “What is this project’s API base URL?”, pure vector search may return several document chunks related to “API” without knowing which project you mean. Graph retrieval can follow the chain “project -> API -> baseUrl” and return the configuration value you set earlier.

That only works if entity extraction is correct. If the LLM splits “API base URL” into two entities, “API” and “base URL,” the graph can fragment and retrieval can break. This is where noise accumulation begins.

Once the positioning is clear, the core capabilities are easier to read.

1.2 Core Capability Matrix

Mnemo’s README lists four core capabilities:

1. persistent knowledge graph

   • Entities and relationships are stored in SQLite, not kept as one-time vectors
   • The graph structure can be queried, exported, and migrated

2. entity extraction

   • Automatically identifies entities from conversations, such as people, project names, API names, and decisions
   • It is not pure vector retrieval. It turns a question like “What is this API for?” into a queryable entity-relationship structure
   • Entity extraction quality depends on the LLM’s understanding. Local LLMs such as llama3 may misidentify entities in complex conversations, and noise accumulation is an ongoing risk. The README does not provide an automatic cleanup system, so you need to inspect graph quality regularly

3. semantic retrieval

   • Combines the graph and vectors, preferring entity relationships over pure similarity during retrieval
   • Reduces vector-search noise: documents with high similarity but irrelevant meaning should not dominate the result

4. graph-first vs pure vector search

Pure vector search asks, “What is similar?” Graph retrieval asks, “What is related?” The first approach can return semantically similar but irrelevant noise. The second can follow relationship chains between entities.

The contrast is clearer in a table:

1.3 Tech Stack and License

Tech stack:

KiloClaw - Managed OpenClaw for enterprise, enabling AI Agent orchestration and smart model routing

Get Started →Ad

• Rust, split into four crates in the next section
• SQLite for local storage, using WAL mode
• petgraph for the in-memory graph
• OpenAI, Ollama, or Anthropic API as LLM backends

License: MIT License, so it can be used, modified, and redistributed.

Risk reminders:

• Early-stage project, based on the 2026-06-05 GitHub README
• APIs and architecture may change
• No broad production validation is documented
• README performance numbers are self-reported, not independently proven

2. Architecture: Four Rust Crates

Mnemo is written in Rust and split into four crates with clear responsibilities:

mnemo-core: core logic

• Entity extraction, graph construction, and retrieval logic
• Does not depend on a specific LLM backend; it defines the interfaces

mnemo-api: server

• Provides the HTTP API on the default port 8080
• Receives conversations, calls core, and returns results
• Health check: curl http://localhost:8080/health

mnemo-cli: command-line tool

• Debugging, management, and querying
• Operates on the local SQLite database directly, without HTTP

mnemo-bench: performance benchmarks

• The README’s 122 Rust tests, 21 Python tests, and 12 benchmarks are reported here
• Source of the self-reported measurements

The four-crate split has advantages:

• core can be tested independently of the API
• cli makes local debugging easier without starting the service
• bench stays separate and does not affect production code

It also has costs:

• You need the full Rust toolchain to compile it, unless you use Docker
• When crate APIs change, multiple places may need to be updated together

3. Installation and Deployment: Three Paths

Mnemo offers three deployment paths, from simplest to more hands-on:

3.1 Docker + Ollama: Fastest First Run

Prerequisites: Docker and Ollama are installed.

# 1. clone the project
git clone https://github.com/zaydmulani09/mnemo.git
cd mnemo

# 2. start Docker
docker compose up -d

# 3. pull a model inside Docker
docker exec mnemo-ollama ollama pull llama3

# 4. health check
curl http://localhost:8080/health

Note: commands may change. Use the GitHub README as the source of truth.

Explanation: Docker compose starts two containers: mnemo-api, the server, and mnemo-ollama, the Ollama service. The latter is optional. If Ollama already runs locally, you can run only the mnemo-api container and connect to your local Ollama with MNEMO_LLM_BASE_URL=http://host.docker.internal:11434/v1.

Verify the LLM connection: when the health check returns {"status":"ok"}, the API is running, but the LLM connection has not been proven yet. Send a test request with curl:

curl -X POST http://localhost:8080/v1/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is this project API base URL?"}'

If the response includes entity-extraction output, the LLM connection is working.

Pros:

Alibaba Cloud - Top choice for developers, exclusive 15% off for Lighthouse servers

Get 15% Discount →Ad

• No Rust toolchain required
• Docker handles dependencies automatically
• Ollama and Mnemo are on the same compose network, so networking is simple

Cons:

• Docker consumes extra resources
• Debugging is less convenient because you need to enter containers to inspect SQLite
• Logs are split across two containers

3.2 Binary: Local Compile

Prerequisites: Rust toolchain is installed, including cargo and rustc, and Ollama is installed.

# 1. clone the project
git clone https://github.com/zaydmulani09/mnemo.git
cd mnemo

# 2. compile the API crate
cargo install --path crates/mnemo-api

# 3. configure the Ollama endpoint
export MNEMO_LLM_BASE_URL=http://localhost:11434/v1

# 4. start the service
mnemo-api

Note: commands may change. Use the GitHub README as the source of truth; this path requires a Rust toolchain.

Explanation: compile time depends on your hardware. On an Apple M2 it is roughly 2 to 3 minutes; Windows and Linux can take longer. If compilation fails, check the Rust version required by the README. After compilation, mnemo-api creates a SQLite file in the current directory by default, usually mnemo.db, and stores the graph there.

Verify the Ollama connection: before starting, confirm that Ollama is running on localhost:11434 and that the model has been pulled with ollama pull llama3. After the service starts, test with curl:

curl -X POST http://localhost:8080/v1/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "test"}'

Pros:

• Does not depend on Docker
• Easier to debug because it is a local process and logs stay in one terminal
• You can use mnemo-cli directly against the local SQLite database
• Port and database path can be customized through environment variables

Cons:

• Full Rust toolchain required
• First compile can take a while
• Dependency management can break, for example when cargo.lock is out of date

3.3 OpenAI-Compatible: Cloud LLM

Prerequisites: you have an API key for OpenAI, Anthropic, or another OpenAI-compatible backend.

Environment variable checklist, verified against the 2026-06 GitHub README:

export MNEMO_LLM_BASE_URL=https://api.openai.com/v1
export MNEMO_LLM_API_KEY=sk-...
export MNEMO_LLM_MODEL=gpt-4o-mini
export MNEMO_LLM_PROVIDER=openai

Then start:

mnemo-api

Note: environment variable names may change. Use the GitHub README as the source of truth.

Best fit:

• Your local machine does not have enough compute, so you use a cloud LLM
• You already have OpenAI API quota
• You accept that conversation content is sent to the cloud API. With a cloud LLM, local-first privacy applies to local storage, not to inference traffic

Choose one of the three paths based on your setup. The next question is performance.

4. Performance: README Self-Reported Numbers

Mnemo’s README lists self-reported performance numbers for the 2026-06-05 version:

Test conditions:

• Apple M2, debug build
• SQLite WAL mode
• petgraph in memory

Performance numbers:

Railway - Modern deployment platform, zero-config, go live in minutes

Start Now →Ad

• Full retrieval pipeline: about 4.2 ms
• Release build claims 3-5x faster, roughly 0.8-1.4 ms

Note: this is README self-testing, not independent proof. Performance changes with hardware, data volume, and LLM backend.

How to read those numbers:

• 4.2 ms is retrieval time, not LLM inference time. LLM inference is the bottleneck
• SQLite WAL plus an in-memory graph can make retrieval fast
• This only measures retrieval, not conversation speed

Actual experience depends on:

• LLM inference time, which is much slower than retrieval
• Conversation length, because entity extraction needs LLM inference
• Data volume, because a larger graph can slow retrieval

Suggestion: run mnemo-bench on your target hardware and use the README numbers only as a reference, not a promise.

5. Local-First Benefits and Boundaries

5.1 Local-First Advantages

The core of local-first is simple: the data stays on your machine.

Specific advantages:

1. Privacy protection

   • Conversations, entities, and relationships are stored in local SQLite
   • They are not uploaded to a third-party SaaS, as long as you use a local LLM

2. Data control

   • The SQLite file can be exported, backed up, and migrated
   • You are not dependent on whether a SaaS vendor survives; your data remains with you

3. Portability

   • When you switch machines, copy the SQLite file
   • You do not need to “train” the memory again

4. Debuggability

   • SQLite is a standard format and can be inspected with any SQLite tool
   • The graph structure is visible instead of being a black-box vector store

5.2 Risks and Boundaries

Local-first has advantages, but it also has risks:

1. Noise accumulation

   • Entity extraction is imperfect and can misidentify terms
   • Misidentified entities affect later retrieval
   • Regular cleanup is needed, but Mnemo does not provide an automatic cleanup system
   • Noise accumulation is not unique to Mnemo; every automatic memory system has it. The difference is that Mnemo’s graph is visible. You can see noisy entities and wrong relationships. Vector-system noise hides inside vectors. That visibility is an advantage, but it is also maintenance work

2. Rollback limits

   • SQLite WAL has rollback capability, but Mnemo does not expose a high-level “undo memory” interface
   • Incorrectly stored memories are awkward to delete manually
   • A worse case is storing an incorrect decision, such as “use Redis for caching.” Later conversations may reason from that wrong decision. Undoing it can require manually deleting every related entity and relationship in the graph, which may be harder than rebuilding the graph
   • Before storing key decisions, verify entity extraction in a test environment

3. Hidden state

   • The graph can contain relationships you do not notice
   • Retrieval may return results without making the path obvious

4. Fit boundaries

   • With large data volumes, SQLite plus an in-memory graph may come under pressure
   • For team collaboration, a local SQLite database is not designed for multiple people writing concurrently

5.3 Fit vs. Skip Decision Table

Judgment: if your scenario is “personal project, high privacy needs, existing Ollama, and modest data volume,” Mnemo is worth a trial. If it is “large data, team collaboration, and global shared memory,” wait for Mnemo to mature or choose another approach.

Concrete suggestions:

• Run the Docker path once in a test environment and inspect the graph structure, entity extraction quality, and retrieval behavior
• Use test conversations to check noise risk. Say intentionally irrelevant things and see whether Mnemo misidentifies them
• Prepare a cleanup plan. Learn the SQLite structure early so you know how to remove wrong entities manually
• Watch README updates. This is an early-stage project, so APIs, architecture, and deployment commands may change

6. Next Step: Series Navigation

If you have not read the earlier articles in the Ollama local LLM series, read them in order:

1. Ollama Beginner Guide: Run a Large Language Model Locally

   • Start here if Ollama is not installed yet

2. Ollama API Calls: From curl to the OpenAI SDK-Compatible Interface

   • Mnemo uses an OpenAI-compatible API, so this helps with the Ollama API surface

3. Ollama Embeddings in Practice: Local Vector Search and RAG

   • Mnemo’s semantic retrieval depends on embeddings, and this article covers the basics

4. AI Agent Memory Management: Long-Term Memory and Knowledge Governance

   • Mnemo is a memory-layer tool; this article covers the governance side of agent memory

After those four, install Mnemo and run the Docker path locally. The useful first result is seeing what the knowledge graph actually looks like.