要点
- On 10 November 2026, NVIDIA published a technical blog, "Start and Restore GPU Workloads in Seconds with NVIDIA Snapshot," by Ron Kahn, Dan Feigin, Oz Bar-Shalom, Ekin Karabulut, and Schwinn Saereesitthipitak. Snapshot is described as an open-source component of the NVIDIA DSX AI Factory Platform, with code at https://github.com/ai-dynamo/snapshot.
- Across the tested configurations, restore took 3.6 to 42.7 seconds, 8x to 36x faster than cold starts that can take minutes. Engine ranges on the post's Table 1 are 4.7 to 39.6 seconds for vLLM, 3.6 to 42.7 seconds for SGLang, and 7.79 to 33.55 seconds for TensorRT-LLM. Multi-GPU checkpoints go up to about 1 TB across eight GPUs.
- Checkpoint creation took 15 seconds to 5.5 minutes, depending on model size. In every tested configuration, checkpoint time plus restore time was shorter than a single cold start. On NVIDIA B200 with VAST NFS storage, restoring GPU state dominated large restores at 18 to 29 seconds, and CRIU-based CPU process restore took 4.2 to 10.8 seconds.
- The cold-start clock runs from process launch until engine initialization returns, before inference, in four phases: startup, load weights, initialization, and compilation plus CUDA-graph capture. The Snapshot clock runs from the restore request until the workload is serving again. Machine provisioning and container image loading sit outside both measurements.
- Restore works only on compatible infrastructure, meaning the same GPU model and driver. The NVIDIA post says Crusoe reduced GLM-5.3 Pod readiness from roughly 8 minutes to 31 seconds on NVIDIA HGX B300, and that Saturn Cloud cut vLLM startup from 2 to 8.5 minutes to 6 to 34 seconds on HGX B300. Crusoe's own post, also dated 10 November 2026, confirms the 31-second figure and adds that it was measured with the checkpoint on local NVMe.
On 10 November 2026, NVIDIA published a technical blog introducing NVIDIA Snapshot, an open-source checkpoint-and-restore system for GPU workloads on Kubernetes. The post, signed by Ron Kahn, Dan Feigin, Oz Bar-Shalom, Ekin Karabulut, and Schwinn Saereesitthipitak, says Snapshot captures fully initialized CPU and GPU memory and brings inference replicas back in seconds instead of minutes. The checkable claim is not that scarce GPUs are no longer a problem. It is that cold-start, the minutes spent loading weights, compiling kernels, and capturing CUDA graphs, is the binding constraint when fleets scale or fail, and that Snapshot restores that initialized state in 3.6 to 42.7 seconds, 8x to 36x versus a cold start, provided the next node has the same GPU model and driver, and provided the clock is allowed to ignore machine provisioning and image pulls.
[1]What Snapshot actually measures
A cold start, in this post, is the work of bringing an already assigned GPU from a freshly launched inference process to a ready engine, before the first request. The authors divide that work into four phases: starting processes and importing libraries, loading model weights, initializing caches and runtime buffers, and compiling kernels, autotuning them, and capturing CUDA graphs. Figure 1 on the post also shows two earlier steps, provisioning the machine and loading the container image, and then says those steps remain outside the measured restore path.
Snapshot was originally built for NVIDIA Dynamo inference workers, and other engines can integrate it. Once a workload is fully initialized, a checkpoint captures its CPU and GPU memory as a portable artifact. A restore starts a new Kubernetes Pod, a scheduled group of containers, from that artifact. The workload needs only a small handshake to pause before the checkpoint and resume after the restore. Ready-made examples exist for vLLM 0.31.0, SGLang 0.5.21, and TensorRT-LLM 1.3.0, the engine versions used in the benchmarks. The system can checkpoint a multi-GPU workload on one node as a single artifact, including tensor parallelism, data and expert parallelism, and decode context parallelism.
The restore clock starts when Snapshot is asked to restore, and it stops when the workload is serving again. That interval has four stages of its own: the node agent locates the artifact and prepares the Pod and GPUs; CRIU, Checkpoint/Restore in Userspace, a standard Linux tool, reads the CPU process image into host memory; CUDA state is copied back into GPU memory; and the process is adapted to the new node and woken. The key-value cache is released before every checkpoint, so it is not part of the stored artifact.
[1][2]The restore times and partner checks
The NVIDIA numbers were collected on NVIDIA B200 GPUs, driver 615.71.09, with VAST NFS storage. Across all tested configurations, Snapshot restored replicas in 3.6 to 42.7 seconds, 8x to 36x faster than a cold start. The post says the largest mixture-of-experts models, with checkpoints of about 600 GB to 1 TB across eight GPUs, restored in under 43 seconds, against cold starts of 6 to 19 minutes. By engine, the ranges are 4.7 to 39.6 seconds for vLLM, 3.6 to 42.7 seconds for SGLang, and 7.79 to 33.55 seconds for TensorRT-LLM. Models in the tables include Qwen3-0.6B, Qwen3-14B, gpt-oss-120b, GLM-5.3-NVFP4, DeepSeek-V4.1-Flash, and DeepSeek-V4-Pro-NVFP4 in a TEP8 setting, meaning tensor and expert parallelism across eight GPUs. TensorRT-LLM was not reported on the two DeepSeek configurations.
Checkpoint creation took 15 seconds to 5.5 minutes, a one-time cost per model and configuration. In every tested configuration, that checkpoint time plus the later restore was still shorter than a single cold start. For the eight-GPU mixture-of-experts configurations, restoring GPU state took 18 to 29 seconds and dominated the large restores, while CRIU-based CPU restore took 4.2 to 10.8 seconds.
Design partners ran their own checks. The NVIDIA post says Crusoe, an upstream contributor, reduced GLM-5.3 Pod readiness from roughly 8 minutes to 31 seconds on NVIDIA HGX B300. Crusoe's own post, also dated 10 November 2026, confirms that 15x comparison and adds a storage detail the NVIDIA post does not emphasize: with the 594 GB checkpoint on local NVMe rather than NFS, GPU-state restore fell from 33.3 seconds to 15.0 seconds, and Pod created-to-Ready fell from 50 seconds to 31 seconds. The headline 31 seconds is therefore the local-NVMe figure. The NVIDIA post also says Saturn Cloud cut vLLM startup from 2 to 8.5 minutes to 6 to 34 seconds on HGX B300. We opened Saturn's post and the ranges match, but those figures are cited here as NVIDIA reported them, except where Crusoe is named as a source.
[1][2]What the clock leaves out
Both the cold-start clock and the Snapshot clock leave out the time to obtain a machine and to pull a container image. If demand spikes and no node is already running, or if the image is not already on the node, the user's wait is longer than the 3.6 to 42.7 second restore. Crusoe's breakdown makes the same point in a different way: even after the agent restore, several seconds of Pod startup sit outside the GPU-state copy, and storage choice moves the total by tens of seconds.
The cold-start phases also show why the speedup can be so large. For some of the biggest configurations, compilation, autotuning, and CUDA-graph capture account for most of the minutes. Snapshot skips those repeated steps by restoring an already initialized engine. That is a real operations gain when a replica dies or when a service scales out, and it is also why the comparison is only fair if both sides start from a process launch on a machine and image that are already there.
[1][2]Same GPU, same driver, and other limits
Restore is limited to compatible infrastructure: the same GPU model and the same driver. A checkpoint taken on a B200 will not, on the evidence in this post, come up on a different GPU or a different driver. The post also requires workloads to meet a published compatibility contract, and the multi-GPU path is for GPUs on a single node. Checkpoint artifacts are large, up to about 1 TB in the tests, so storage bandwidth becomes part of the restore time, as Crusoe's NFS versus NVMe comparison shows.
These are NVIDIA's and its partners' measurements, not a third-party audit of a running production fleet. Saturn's post, which we opened, is careful to say its numbers are worker-Pod-to-first-response measurements on an already-running node, from a small number of paired runs, and not the full wait when a GPU node must also be provisioned. That caveat applies equally to the NVIDIA tables.
[1][2]The fair case
The fair reading is that skipping minutes of weight loading, compilation, and graph capture is a real operations win when demand spikes or a replica dies, and that partner times point the same way. In every configuration NVIDIA tested, even the one-time cost of writing the checkpoint plus one restore was cheaper, in wall-clock, than starting from scratch once. For operators who already keep a compatible node warm, or who can place the artifact on fast local storage, the 31-second and sub-45-second figures are the right comparison class.
The honest limit sits beside that finding. The headline speedup does not include getting the machine and the container image, and it will not help if the next node has a different GPU or driver. Storage backend and cache warmth also move the result, as Crusoe shows by placing the same 594 GB artifact on NFS and on NVMe. Snapshot changes the restart of an initialized engine. It does not, by itself, change the time to acquire the GPU.
[1][2]Who should care, and what to watch
Platform teams running vLLM, SGLang, or TensorRT-LLM on Kubernetes, and operators who today keep spare replicas warm because a cold start is too slow, should care. The code is open at the NVIDIA Snapshot repository, and the post says other engines can integrate. In the next six months, the tests are whether restore still works across driver upgrades and new GPU models, whether engines beyond the three named examples ship a supported integration, and whether operators publish production scale-from-zero times that include, rather than exclude, node provisioning and image pulls.
[1][2]Close
Snapshot makes an initialized GPU worker reusable, and the measured restore times are short enough to matter when a replica fails or traffic jumps. The comparison only holds on the same GPU and driver, and only after the machine and the image are already in place. Those are the conditions to keep attached to the seconds, as fleets start to rely on them.
[1]