I Built a RAG Search Engine from Scratch to Understand How Modern Search Actually Works

Source: Dev.to

Overview

Everyone is building RAG apps, but most tutorials skip the most important part — search quality.
Instead of just plugging in a framework, I built my own RAG Search Engine from scratch to deeply understand how retrieval systems work under the hood.

You can watch the full breakdown here:

• YouTube Video:
• Source Code:

Retrieval Techniques Implemented

Classical Keyword Search

• Term Frequency (TF)
• Inverse Document Frequency (IDF)
• BM25 scoring

These techniques illustrate why traditional keyword search remains extremely powerful in production systems.

Dense Semantic Retrieval

• Embedding‑based representations
• Cosine similarity for meaning‑based matching

Enables handling of:

• Synonyms
• Contextual variations
• Conceptual similarity

Hybrid Ranking

Combined keyword and semantic signals using:

• Weighted fusion
• Reciprocal Rank Fusion (RRF)

This mirrors modern production search systems that blend precision with semantic understanding.

Reranking Stage

After retrieving the top results, a reranking model evaluates query‑document pairs more deeply, significantly improving relevance and precision.

Evaluation Metrics

• Precision
• Recall
• F1 Score
• Manual evaluation
• LLM‑as‑a‑judge evaluation

These metrics provide a comprehensive view of retrieval performance.

Multimodal Retrieval

Experimented with text + image retrieval using embedding‑based similarity, extending the engine beyond pure text search.

Integration with LLM (RAG)

Connected the hybrid retrieval system to a large language model to generate grounded responses.
Key takeaway: Better retrieval > Bigger model.

Lessons Learned

• Retrieval quality is the hardest and most critical part of RAG pipelines.
• Hybrid systems consistently outperform pure keyword or pure semantic approaches.
• Reranking adds a substantial boost to precision.
• Proper evaluation (both quantitative metrics and qualitative judgment) is essential for trustworthy results.
• Designing search systems involves trade‑offs between latency, index size, and relevance.

Topics of Interest

If you’re interested in any of the following, let’s discuss:

• Search engines
• Information retrieval
• RAG systems
• AI system design
• Hybrid search architectures

I’d love to hear your thoughts—what would you improve or explore next? 👇