โ€œ๐‡๐จ๐ฐ ๐†๐จ๐จ๐ ๐ฅ๐ž ๐ฌ๐ก๐จ๐ฐ๐ฌ ๐ฒ๐จ๐ฎ ๐ฌ๐ž๐š๐ซ๐œ๐ก ๐ซ๐ž๐ฌ๐ฎ๐ฅ๐ญ๐ฌ ๐ข๐ง ๐ŸŽ.๐Ÿ‘ ๐ฌ๐ž๐œ๐จ๐ง๐๐ฌ (๐ญ๐ก๐ž๐ฒโ€™๐ซ๐ž ๐œ๐ก๐ž๐š๐ญ๐ข๐ง๐ , ๐š๐ง๐ ๐ฒ๐จ๐ฎ ๐ฌ๐ก๐จ๐ฎ๐ฅ๐ ๐ญ๐จ๐จ)โ€

Published: (December 9, 2025 at 07:12 PM EST)
5 min read
Source: Dev.to

Source: Dev.to

Hmโ€ฆ I think internet cannot be searched; you only look up on Google, right?

You type โ€œbest jollof rice in Lagosโ€ into Google. Three hundred milliseconds later, you have ten perfect results. In that time, Google supposedly searched through hundreds of billions of web pages, ranked them by relevance, personalized results based on your location and history, and sent everything back to your phone.
Except they didnโ€™t. Thatโ€™s physically impossible. Hereโ€™s what actually happened, and why it changes everything about how you should build fast applications.

When you press search, youโ€™re not actually searching the internet. Youโ€™re searching Googleโ€™s preโ€‘computed index of the internet, which is a completely different thing.

Imagine a library with one million books and someone asks you to find every book that mentions โ€œjollof rice.โ€ Scanning every page would take days or weeks. Instead, you could spend months beforehand creating an indexโ€”a massive book that lists every word that appears in the library and exactly which books and pages contain that word.

When someone asks for โ€œjollof rice,โ€ you just open the index to the โ€œjollofโ€ section, see the list of books and page numbers, do the same for โ€œrice,โ€ find the books that appear in both lists, and hand them the answer in seconds. You did the hard work before anyone asked the question.

Thatโ€™s exactly what Google does, just at planetary scale.

Letโ€™s break down the steps that occur in those three hundred milliseconds:

  1. Frontโ€‘end servers (GFE) โ€“ Your query hits Googleโ€™s globally distributed frontโ€‘end servers, located close to you (e.g., a West African GFE if youโ€™re in Lagos).
  2. Spelling correction โ€“ Google checks for typos and autoโ€‘corrects if needed. This runs on Googleโ€™s internal infrastructure called Borg (the precursor to Kubernetes).
  3. Index shards โ€“ The corrected query is sent to Googleโ€™s index shards. The index is split into thousands of shards distributed across data centers worldwide. Each shard holds a portion of the inverted index plus the actual document data.
  4. Topโ€‘k fetch โ€“ Each relevant shard quickly fetches the top ~1,000 possible documents that match the query. The index is already sorted and optimized for this lookup.
  5. Ranking โ€“ All those results are sent to Googleโ€™s ranking system, which uses over two hundred signals (location, search history, freshness, backlinks, mobileโ€‘friendliness, page speed, etc.) to sort them in milliseconds.
  6. Formatting โ€“ The top ten results are formatted and sent back to your browser. The whole process takes under half a second.

Critical insight: Google isnโ€™t searching billions of pages in real time; itโ€™s looking up preโ€‘computed results in an index that is continuously updated in the background.

The inverted index: Googleโ€™s secret weapon

An inverted index stores words and the list of documents that contain them, rather than storing whole documents.

Normal storage
Document 1 contains โ€œI love jollof riceโ€

Inverted index

  • โ€œjollofโ€ appears in: Documentโ€ฏ1, Documentโ€ฏ2
  • โ€œriceโ€ appears in: Documentโ€ฏ1
  • โ€œLagosโ€ appears in: Documentโ€ฏ2

When someone searches for โ€œjollof Lagos,โ€ the system instantly knows Documentโ€ฏ2 is the only one containing both wordsโ€”no document scanning required.

Googleโ€™s inverted index tracks hundreds of billions of web pages and trillions of words. Because it is preโ€‘computed and sharded across thousands of servers, lookups are extremely fast.

Incremental indexing: How Google stays current

The web changes constantlyโ€”new pages appear, old pages update, sites go offline. Rebuilding the entire index from scratch each time would be impossible.

Google uses incremental indexing:

  • Crawlers continuously browse the web, looking for new or changed content.
  • When a change is detected, only the relevant portions of the index are updated.
  • Popular news sites are crawled every few minutes; smaller sites may be crawled every few days or weeks.

The index is never perfectly upโ€‘toโ€‘date, but itโ€™s current enough that users donโ€™t notice. Breaking news can appear in search results within minutes; a small blog post might take hours or days.

Why 99โ€ฏ% of apps are slow

Most applications are slow because they perform full database scans or complex calculations on every request. Googleโ€™s approach is the opposite: preโ€‘compute everything you possibly can.

Example: Foodโ€‘delivery app

Slow approach

  1. User opens app.
  2. App gets userโ€™s location.
  3. App queries the database: โ€œFind all restaurants, calculate distance, filter within 5โ€ฏkm, sort by rating.โ€
  4. Database scans thousands of rows, performs distance calculations, and returns results after a few seconds.

Googleโ€‘style approach

  1. Every few minutes, preโ€‘compute which restaurants are near every major neighborhood.
  2. Store these lists in fast memory (e.g., Redis).
  3. When the user opens the app, look up the preโ€‘computed list for the relevant neighborhood.
  4. Results appear in a few hundred milliseconds.

The expensive work (distance calculations, filtering, sorting) is moved to background jobs, not request time.

Real examples where preโ€‘computation matters

  • YouTube recommendations โ€“ Algorithms run continuously in the background, updating recommendation lists for hundreds of millions of users. When you open the app, you see a list that was preโ€‘computed minutes or hours ago.
  • Instagram feed โ€“ The feed is assembled in the background based on who you follow and your engagement history. Pullโ€‘toโ€‘refresh simply fetches the preโ€‘built feed.
  • Eโ€‘commerce search (Amazon) โ€“ Inverted indexes of products by keywords, categories, and attributes allow instant lookups instead of scanning the entire product catalog.
  • Banking dashboards โ€“ Account balances and transaction histories are aggregated and cached for dashboard views; the main transaction database handles new writes, while reporting views are preโ€‘computed.

The tradeโ€‘off: freshness vs. speed

Preโ€‘computation introduces a single major tradeโ€‘off: the data may be slightly stale. If Googleโ€™s index was updated ten minutes ago and a website changed five minutes ago, the search results wonโ€™t reflect that change yet.

For most use cases, this is acceptable. Users expect fast responses, not perfect realโ€‘time accuracy. The slight lag is a small price to pay for the dramatic performance gains that preโ€‘computed indexes provide.

Back to Blog

Related posts

Read more ยป