Under the Hood: Building a Hybrid Search Engine for AI Memory (Node.js + pgvector)

Source: Dev.to

1. The Core Philosophy: Pragmatism

The architecture is designed to reduce infrastructure cognitive load by avoiding a separate vector database. All vectors and metadata live in PostgreSQL with the pgvector extension, preserving ACID compliance and simplifying the stack.

• Runtime: Node.js (event‑driven I/O for orchestrating DB/LLM calls)
• Language: TypeScript (strict typing for 1536‑dimensional float arrays)
• Storage: PostgreSQL + pgvector (vectors + session_id, user_id in the same engine)

2. The Hybrid Search Algorithm

MemVault computes a weighted score in real time:

Score = (SemanticSimilarity * α) + (RecencyScore * β) + (Importance * γ)

Components

• Semantic Similarity (α) – Cosine distance calculated by pgvector. Measures topical match.
• Recency Decay (β) – A decay function applied to the timestamp so older memories fade unless reinforced.
• Importance (γ) – An explicit weight for facts that should never decay (e.g., “User is allergic to nuts”).

Typical defaults: 80 % Semantic, 20 % Recency. Adjust α, β, γ to tune agent behavior.

3. Observability: The “Black Box” Problem

Vector search is opaque; debugging hallucinations can be difficult. MemVault includes a real‑time visualizer that projects high‑dimensional vectors onto a 2‑D graph, allowing you to inspect clusters and spot embedding issues.

4. Open Source & Roadmap

MemVault is fully open source. The next roadmap milestone is adding BM25 (keyword search) to handle identifiers (e.g., product IDs) where pure semantic search struggles.

Project Links

• GitHub Repository: https://github.com/jakops88-hub/Long-Term-Memory-API
• NPM Package: https://www.npmjs.com/package/memvault-sdk-jakops88
• Live Demo: https://memvault-demo-g38n.vercel.app/
• License: MIT

5. Quick Start

Choose between self‑hosting (Docker) or a managed API (RapidAPI).

Getting Started

# Clone the repo
git clone https://github.com/jakops88-hub/Long-Term-Memory-API.git
cd Long-Term-Memory-API

# Build and run with Docker
docker compose up -d

Then install the SDK:

npm install memvault-sdk-jakops88

Use the SDK to store chunks, embed them, and query with the hybrid score:

import { MemVault } from 'memvault-sdk-jakops88';

const client = new MemVault({ connectionString: process.env.DATABASE_URL });

await client.storeChunk({
  session_id: 'abc123',
  user_id: 'user42',
  content: 'User is allergic to peanuts',
  importance: 1.0, // never decays
});

const results = await client.search({
  query: 'What can I eat?',
  topK: 5,
  alpha: 0.8,
  beta: 0.2,
  gamma: 0.0,
});

Feel free to open issues or submit pull requests if you have questions about the pgvector implementation or want to contribute to the hybrid scoring logic!