AI Infrastructure Races Ahead Despite Slowdown Calls

While debate raged at Salesforce's Dreamforce over slowing AI deployment, tech leaders at the AI Infra Summit in Santa Clara were focused on building AI infrastructure faster. Experts from AWS, Qualcomm, Oracle, Broadcom, and d-Matrix outlined efforts to meet surging inference demands, with tokens per watt emerging as the defining metric. Memory bottlenecks are a central challenge, with AI server memory spend projected to surge from $35 billion in 2025 to nearly $190 billion by 2027. Qualcomm is betting on its proprietary high-bandwidth compute architecture, while startup d-Matrix is pursuing 3D DRAM through its Raptor chip, now integrated into Nvidia's NVLink Fusion rack architecture.