GPU wiring and power cause massive AI latency swings

CoreWeave SVP Lukas Biewald says GPU networking and power distribution can swing AI latency by orders of magnitude, a finding that surprised him after joining CoreWeave through its acquisition of Weights and Biases. CoreWeave launched CoreWeave Forge, a unified layer for training, inference, evaluation and agent development. LlamaIndex, which processes millions of document pages daily for finance, legal and insurance clients, runs 75% inference workloads and relies on CoreWeave for burst capacity without owning hardware. CoreWeave differentiates itself from hyperscalers by following open Nvidia networking standards rather than proprietary APIs.