Databricks Agent Beats Stronger Models by 21%

Databricks researchers found that multi-step AI agents outperform single-turn RAG systems by 20% or more on hybrid data tasks, even when the baseline uses a stronger model. The gains were measured across nine enterprise knowledge tasks on Stanford's STaRK benchmark and Databricks' own KARLBench framework. Databricks argues the performance gap is an architectural problem, not a model quality problem. The work extends the company's earlier instructed retriever research on metadata-aware queries for unstructured data retrieval.