SageMaker HyperPod Now Caches Models to Cut Cold Starts

Amazon SageMaker HyperPod now supports model caching, a feature that pre-loads model weights and container images onto cluster nodes before pods need them. Instead of downloading over the network, pods read from local NVMe storage at roughly 7 GB/s, reducing startup times from 30-plus minutes to seconds for large models like DeepSeek-R1. The feature introduces two independent capabilities: a weights cache that downloads model data to local NVMe ahead of time, and an image cache that pre-pulls container images to skip ECR downloads. Both use preferred scheduling, meaning pods are never blocked from starting even if a warm cache is unavailable on a given node.