AI 스캐폴딩 레이어가 붕괴하고 있다. LlamaIndex CEO가 살아남는 것이 무엇인지 설명한다.

Source: VentureBeat

Context is becoming the moat

Liu’s LlamaIndex is one of the foremost retrieval‑augmented generation (RAG) frameworks connecting private, custom, and domain‑specific data to LLMs. Even he acknowledges that these types of frameworks are becoming less relevant.

• With every new release, models demonstrate incremental capabilities to reason over “massive amounts” of unstructured data, and they’re getting better at it than humans.
• They can be trusted to reason extensively, self‑correct, and perform multi‑step planning. Modern Context Protocol (MCP) and Claude Agent Skills plug‑ins allow models to discover and use tools without requiring integrations for each one independently.
• Agent patterns have consolidated toward what Liu calls a managed agent diagram—a harness layer combined with tools, MCP connectors, and skills plug‑ins, rather than custom‑built orchestration for every workflow.

Coding agents now excel at writing code, meaning developers don’t need to rely on extensive libraries. Liu notes that about 95 % of LlamaIndex code is generated by AI:

“Engineers are not actually writing real code. They’re all typing in natural language.”

This collapses the layers between programmers and non‑programmers, because “the new programming language is essentially English.” Instead of manual coding or struggling with API and document integration, developers can simply point Claude Code at the data. Liu adds:

“This type of stuff was either extremely inefficient or would break the agent three years ago. It’s just way easier for people to build even relatively advanced retrieval with extremely simple primitives.”

Core differentiator: context

When the stack collapses, context becomes the key differentiator. Agents need to decipher file formats to extract the right information. Providing higher accuracy and cheaper parsing is crucial, and LlamaIndex is well‑positioned thanks to its developments in agentic document processing via optical character recognition (OCR).

“We’ve really identified that there’s a core set of data that has been locked up in all these file format containers. Ultimately, whether you use OpenAI Codex or Claude Code doesn’t really matter. The thing that they all need is context.”

Keeping stacks modular

There’s growing concern about builders like Anthropic locking in session data. Liu emphasizes the importance of modularity and agnosticism:

• Builders shouldn’t bet on any one frontier model or overbuild in a way that overcomplicates components of the stack.
• Retrieval has evolved into an “agent‑plus‑sandbox” model, and enterprises must keep their code bases free of tech debt and adaptable to changing patterns.
• Some parts of the stack will eventually need to be discarded as new models emerge.

“Because with every new model release, there’s always a different model that is kind of the winner. You want to make sure you actually have some flexibility to take advantage of it.”

Podcast highlights

Listen to the Beyond the Pilot podcast for more details, including:

• LlamaIndex’s beginnings as a “toy project” with initially only about 40 % accuracy.
• How SaaS companies can tap into complicated workflows that must be standardized and repeatable for average knowledge workers.
• Why vertical AI companies are taking off and why “build versus buy” remains a very valid question in the agent age.

You can also listen and subscribe to Beyond the Pilot on Spotify, Apple Podcasts, or wherever you get your podcasts.