โ€œ๐‡๐จ๐ฐ ๐†๐จ๐จ๐ ๐ฅ๐ž ๐ฌ๐ก๐จ๐ฐ๐ฌ ๐ฒ๐จ๐ฎ ๐ฌ๐ž๐š๐ซ๐œ๐ก ๐ซ๐ž๐ฌ๐ฎ๐ฅ๐ญ๐ฌ ๐ข๐ง ๐ŸŽ.๐Ÿ‘ ๐ฌ๐ž๐œ๐จ๐ง๐๐ฌ (๐ญ๐ก๐ž๐ฒโ€™๐ซ๐ž ๐œ๐ก๐ž๐š๐ญ๐ข๐ง๐ , ๐š๐ง๐ ๐ฒ๐จ๐ฎ ๐ฌ๐ก๐จ๐ฎ๐ฅ๐ ๐ญ๐จ๐จ)โ€

Published: (December 9, 2025 at 07:12 PM EST)
5 min read
Source: Dev.to

Source: Dev.to

When you press search, youโ€™re not actually searching the internet. Youโ€™re searching Googleโ€™s preโ€‘computed index of the internet, which is a completely different thing.

Imagine a library with one million books and someone asks you to find every book that mentions โ€œjollof rice.โ€ Scanning every page would take days or weeks. Instead, you could spend months beforehand creating an indexโ€”a massive book that lists every word that appears in the library and exactly which books and pages contain that word.

When someone asks for โ€œjollof rice,โ€ you just open the index to the โ€œjollofโ€ section, see the list of books and page numbers, do the same for โ€œrice,โ€ find the books that appear in both lists, and hand them the answer in seconds. You did the hard work before anyone asked the question.

Thatโ€™s exactly what Google does, just at planetary scale.

Letโ€™s break down the steps that occur in those three hundred milliseconds:

  1. Frontโ€‘end servers (GFE) โ€“ Your query hits Googleโ€™s globally distributed frontโ€‘end servers, located close to you (e.g., a West African GFE if youโ€™re in Lagos).
  2. Spelling correction โ€“ Google checks for typos and autoโ€‘corrects if needed. This runs on Googleโ€™s internal infrastructure called Borg (the precursor to Kubernetes).
  3. Index shards โ€“ The corrected query is sent to Googleโ€™s index shards. The index is split into thousands of shards distributed across data centers worldwide. Each shard holds a portion of the inverted index plus the actual document data.
  4. Topโ€‘k fetch โ€“ Each relevant shard quickly fetches the top ~1,000 possible documents that match the query. The index is already sorted and optimized for this lookup.
  5. Ranking โ€“ All those results are sent to Googleโ€™s ranking system, which uses over two hundred signals (location, search history, freshness, backlinks, mobileโ€‘friendliness, page speed, etc.) to sort them in milliseconds.
  6. Formatting โ€“ The top ten results are formatted and sent back to your browser. The whole process takes under half a second.

Critical insight: Google isnโ€™t searching billions of pages in real time; itโ€™s looking up preโ€‘computed results in an index that is continuously updated in the background.

The inverted index: Googleโ€™s secret weapon

An inverted index stores words and the list of documents that contain them, rather than storing whole documents.

Normal storage

Document 1 contains โ€œI love jollof riceโ€

Inverted index

  • โ€œjollofโ€ appears in: Documentโ€ฏ1, Documentโ€ฏ2
  • โ€œriceโ€ appears in: Documentโ€ฏ1
  • โ€œLagosโ€ appears in: Documentโ€ฏ2

When someone searches for โ€œjollof Lagos,โ€ the system instantly knows Documentโ€ฏ2 is the only one containing both wordsโ€”no document scanning required.

Googleโ€™s inverted index tracks hundreds of billions of web pages and trillions of words. Because it is preโ€‘computed and sharded across thousands of servers, lookups are extremely fast.

Incremental indexing: How Google stays current

The web changes constantlyโ€”new pages appear, old pages update, sites go offline. Rebuilding the entire index from scratch each time would be impossible.

Google uses incremental indexing:

  • Crawlers continuously browse the web, looking for new or changed content.
  • When a change is detected, only the relevant portions of the index are updated.
  • Popular news sites are crawled every few minutes; smaller sites may be crawled every few days or weeks.

The index is never perfectly upโ€‘toโ€‘date, but itโ€™s current enough that users donโ€™t notice. Breaking news can appear in search results within minutes; a small blog post might take hours or days.

Why 99โ€ฏ% of apps are slow

Most applications are slow because they perform full database scans or complex calculations on every request. Googleโ€™s approach is the opposite: preโ€‘compute everything you possibly can.

Example: Foodโ€‘delivery app

Slow approach

  1. User opens app.
  2. App gets userโ€™s location.
  3. App queries the database: โ€œFind all restaurants, calculate distance, filter within 5โ€ฏkm, sort by rating.โ€
  4. Database scans thousands of rows, performs distance calculations, and returns results after a few seconds.

Googleโ€‘style approach

  1. Every few minutes, preโ€‘compute which restaurants are near every major neighborhood.
  2. Store these lists in fast memory (e.g., Redis).
  3. When the user opens the app, look up the preโ€‘computed list for the relevant neighborhood.
  4. Results appear in a few hundred milliseconds.

The expensive work (distance calculations, filtering, sorting) is moved to background jobs, not request time.

Real examples where preโ€‘computation matters

  • YouTube recommendations โ€“ Algorithms run continuously in the background, updating recommendation lists for hundreds of millions of users. When you open the app, you see a list that was preโ€‘computed minutes or hours ago.
  • Instagram feed โ€“ The feed is assembled in the background based on who you follow and your engagement history. Pullโ€‘toโ€‘refresh simply fetches the preโ€‘built feed.
  • Eโ€‘commerce search (Amazon) โ€“ Inverted indexes of products by keywords, categories, and attributes allow instant lookups instead of scanning the entire product catalog.
  • Banking dashboards โ€“ Account balances and transaction histories are aggregated and cached for dashboard views; the main transaction database handles new writes, while reporting views are preโ€‘computed.

The tradeโ€‘off: freshness vs. speed

Preโ€‘computation introduces a single major tradeโ€‘off: the data may be slightly stale. If Googleโ€™s index was updated ten minutes ago and a website changed five minutes ago, the search results wonโ€™t reflect that change yet.

For most use cases, this is acceptable. Users expect fast responses, not perfect realโ€‘time accuracy. The slight lag is a small price to pay for the dramatic performance gains that preโ€‘computed indexes provide.

Back to Blog

Related posts

Read more ยป