要点
- AWS enables Seekable OCI (SOCI) indexing for SageMaker training/processing so images lazy-load without job-definition changes.
- Benchmarks: standard pulls grow to minutes with image size; lazy-load stays ~3–11s (~8×–46×); ~34× on a 13.2GB image across 13 instance configs.
- Full training: first epoch +3–16s only; later epochs match; GPU idle during image pull cut by up to ~90% in AWS figures.
Deep-learning images routinely exceed 10GB. On SageMaker, billable job time includes minutes spent downloading and unpacking that image—worst for tuning sweeps and short jobs. AWS now supports Seekable OCI (SOCI) indexing for training and processing, moving full image download off the critical path.
The thesis is concrete: many teams think they are waiting on GPUs when they are paying an image-pull tax. If lazy-load does not harm later epoch throughput, start-path engineering is the first lever on iteration speed.
[1]SOCI is open-source seekable OCI: the runtime fetches only files needed to launch, then streams the rest on access. The index is a separate OCI artifact in ECR linked to the original image, so layers stay unchanged. Users point at pre-indexed AWS Deep Learning Containers or add an index to their own image—without changing job definition, entry point, or data channels.
[1]AWS reports: across eight image sizes, standard Docker pull time grows with size while SOCI stays ~3–11 seconds (~8×–46×). For a 13.2GB image across 13 instance type/size combos, standard pulls took 3m16s–4m20s versus 6–8s lazy (~34×).
In full ResNet-50/Imagenette and Qwen2.5-0.5B SFT jobs, per-epoch time matched from epoch two onward; epoch one ran 3–16s longer while files faulted in. GPU idle during image pull fell by up to ~90% in their charts.
[1]Steelman: multi-hour jobs shrink the relative win; workloads that touch nearly every layer still pay network later; shared caches may already blunt cold starts.
Vendor benches lack independent replication and adversarial access patterns; index-build cost for custom giant images is thinly disclosed. Confidence high on mechanism and order-of-magnitude; medium on fleet-wide dollar savings.
[1]Platform teams should treat image mass as a product metric again. Research iteration—short jobs and wide sweeps—benefits first. Once start tax falls, algorithm gains can be compared more honestly to infrastructure gains.
[1]Watch pre-indexed DLC coverage, frictionless indexing for custom images, and whether SOCI still dominates wall-clock when data channels are heavy. If yes, lazy container start stops being something teams endure and becomes a default requirement.
[1]