A Second Brain That My AI and I Share

Source: Dev.to

Introduction

My AI assistant, Nabu, and I share the same brain—literally. We both read and write to the same org‑roam knowledge base. When I update a project note in Emacs, Nabu sees the change instantly. When Nabu learns something, it writes the information back to the knowledge base, where I can see it in my daily file. This post explains how the system works.

Nabu’s Integration with Org‑roam

Nabu connects to my self‑hosted Matrix server and accesses the org‑roam database through a custom MCP (Matrix Client Protocol) server I built. The integration follows a simple rule set:

• Search first – Before answering any question about a project, person, or decision, Nabu searches org‑roam. This prevents hallucinations and ensures factual answers.
• Record learning – When Nabu learns something worth keeping (e.g., a decision or a fix), it writes a note to the knowledge base.
• Shared memory – Context persists across conversations, giving both the AI and me a continuous shared memory.

Example Interactions

• While writing this post, I asked Nabu to find examples of our shared brain in action. It searched the daily notes and memory files, returning the very interaction we’re having now.
• When troubleshooting a camera‑alert system, I didn’t need to explain the setup. Nabu located the Camera Alerts Pipeline note (camera ID mapping, MQTT topics, container names, alert rules) and fixed the configuration automatically.
• When my Home Assistant dashboard became stale, Nabu retrieved sensor names, device configurations, and integration details from the notes and updated the dashboard.

Agent Instructions

In the OpenClaw framework, the agent reads workspace files at startup. Two key files define its behavior:

AGENTS.md

## Org-Roam Knowledge Base

Org-roam is your primary knowledge base. Search it before answering
questions about projects, people, or decisions. Update it when you
learn something worth keeping.

MEMORY.md

## Org-Roam Knowledge Base: My Primary Role

I am the live interface for Don's org-roam second brain. I can read,
search, create, edit, and reorganize notes.

These instructions tell the agent where to look and what actions it may perform.

Why I Chose Org‑roam and Emacs

1. Plain‑text files – No proprietary database, no cloud lock‑in. Files can be read with cat, grepped, and version‑controlled with Git. LLMs can ingest them directly.
2. Backlinks & graph – Org‑roam adds automatic backlinks and a graph structure, allowing connections to emerge organically.
3. Programmability – Emacs can be extended with Lisp to fit any workflow.
4. Data sovereignty – All notes reside on my machine and sync via Git to my own server; no third‑party copy exists.

Components of the Shared Brain

• Enhanced org‑roam package – Adds structured node types (people, projects, ideas, admin tasks) with dedicated templates, semantic search via vector embeddings stored in the org files, and proactive surfacing (daily digests, stale‑item alerts, dangling links).
• MCP server – Exposes ~30 JSON‑RPC tools (semantic, contextual, keyword search; CRUD; task state management; surfacing). It runs locally on port 8001, allowing any HTTP‑capable tool—including AI agents—to interact with the knowledge base.

Example JSON‑RPC Call

curl -X POST http://localhost:8001 \
  -H "Content-Type: application/json" \
  -d '{
        "jsonrpc":"2.0",
        "id":1,
        "method":"tools/call",
        "params":{
          "name":"semantic_search",
          "arguments":{"query":"container networking"}
        }
      }'

Replicating the Setup

1. Install the Emacs package (via straight.el):

   (straight-use-package
    '(org-roam-second-brain :host github :repo "dcruver/org-roam-second-brain"))
   
   (require 'org-roam-second-brain)
   (require 'org-roam-api)

2. Run an embedding server (choose one):

   • Infinity (Docker)

     docker run -d -p 8080:7997 michaelf34/infinity:latest \
       --model-id nomic-ai/nomic-embed-text-v1.5

   • Ollama

     ollama pull nomic-embed-text

3. Start the MCP server:

   pip install org-roam-mcp
   export EMACS_SERVER_FILE=~/emacs-server/server
   org-roam-mcp --port 8001

4. Configure your AI agent to treat org‑roam as its source of truth. Details depend on the agent framework; refer to the full setup guide for specifics.

Considerations

This configuration is best suited for users comfortable with Emacs, willing to run local services, and prepared for some initial configuration effort. The payoff is a self‑hosted, permanent knowledge base searchable by meaning rather than just keywords, enabling a truly shared context between human and AI.

Resources

• GitHub repository: https://github.com/dcruver/org-roam-second-brain
• OpenClaw integration: https://github.com/dcruver/openclaw

Feel free to take what’s useful and ignore the rest.