Large Language Model in search engines

Source: Dev.to

A* Algorithm Based on Adaptive Weights

The A* algorithm uses a heuristic + the total cost up to a node n.
It works well on a normal graph, but when the search space is a grid it shows a few flaws.

In a grid‑based search space the graph is an n × n array of cells, each cell being a node.
Moving from one cell to an adjacent cell has a constant cost throughout the grid.

Below are two main problems of the traditional A* algorithm on a grid and the proposed solutions.

Problem 1 – Too Many Neighbours Considered

When running traditional A* on an n × n grid we usually consider 8 neighbours (the 8‑connected neighbourhood).
Exploring all eight neighbours at every iteration wastes computational power by expanding unnecessary nodes.

Proposed solution: consider only the 5 most useful neighbours.
The question is: how do we pick those 5 neighbours?

Step 1 – Compute Δx and Δy

[ \Delta x = |x_{\text{goal}} - x_{\text{current}}|,\qquad \Delta y = |y_{\text{goal}} - y_{\text{current}}| ]

Step 2 – Choose the appropriate condition

Compare the two values with the conditions below:

Step 3 – Select the neighbour set

Whichever condition is satisfied, take the corresponding set of neighbours.
The format for the neighbour set is shown in the figure (the blue cell is the agent).

Problem 2 – Hazardous Paths

A hazardous path is one that passes directly across the edge of an obstacle (see the figure).
When the traditional A* algorithm expands such a path, the agent can collide with the sharp edge of the obstacle.

Proposed solution: remove two neighbours from the neighbour set based on the direction of the obstacle.
The table below shows which child nodes must be excluded for each obstacle direction.

The algorithm checks the obstacle’s direction and then decides which child nodes should not be considered.

Personal Thoughts

I have been learning about A* and other path‑finding algorithms in class recently, and I find them fascinating.
Reading the paper “A Algorithm Based on Adaptive Weights”* gave me concrete ideas on how to implement these concepts in real‑world scenarios.

Reference paper:
A* algorithm Based on Adaptive Weights

Large Language Models in Search Engines

Modern search engines have become highly autonomous, making daily tasks more productive.
Earlier search engines relied on pre‑programmed keyword matching and classical Natural Language Processing (NLP) techniques.

The old problem

Complex queries such as

“How to know if and A* algorithm has given optimal path should we compare it with ucs traversal?”

confused traditional engines because:

• The sentence contains grammatical errors (e.g., the stray “and”).
• Ambiguous terms like “ucs” were not understood.

The modern solution

We now combine Large Language Models (LLMs) (e.g., ChatGPT, Claude, Opus) with web technologies.

• LLMs excel at understanding human language using the latest NLP advances.
• However, they lack up‑to‑date factual knowledge.

By integrating LLMs with live web retrieval (RAG – Retrieval‑Augmented Generation), we obtain a search engine that:

1. Interprets complex, noisy queries accurately.
2. Retrieves the most recent information from the web.

Evolution of Search Engines

Section 3: The Deep Search

Now comes the latest and most powerful method to date: THE DEEP SEARCH.
In this approach, instead of following a fixed step‑by‑step procedure, the agent is allowed to think independently. The agent decides what to search for and determines the best content to deliver after understanding the user’s complex query. Machine learning is also employed in this cutting‑edge technology.

Personal thoughts

Before reading this paper I didn’t have any idea about the history of search engines. I learned a lot about how search engines work and how they have evolved over the decades.

Reference papers

• A Survey on AI Search with Large Language Models
• Video link
• @raqeeb_26 (dev.to)