要点
- AWS ML Blog (2026-11-03) shows sequence-level knowledge distillation (SeqKD) on Amazon SageMaker AI that compresses GLM-5.3 teacher behavior (on Bedrock) into a Qwen3.5-4B student.
- ShoppingBench: 750 problems → 426 train / 324 held-out; teacher sampled twice per problem at T=1.0 (850 trajectories); the bench scorer keeps only solved runs; per-step encoding ≈2,600 train rows.
- Held-out Absolute Success Rate: base 4B avg 0.297 → SeqKD student 0.541; greedy teacher 0.540, sampled teacher 0.511, medium frontier 0.546. Tool-calls-only (reasoning stripped) → 0.415 (~−13 pts).
- IFEval prompt-strict 0.819→0.741 and instruction-strict 0.872→0.813 (~−6–8 pts)—specialization has a regression cost; authors call SeqKD a first step, with on-policy methods for the hard tail.
You ship a shopping assistant that works: a shopper asks for “a black waist bag under twenty dollars from a shop with at least 4.5 stars,” and the agent searches, checks constraints, and recommends. The pain is not one call—it is the billed loop. Search, read, refine, verify, decide: each step can be another frontier invoice. AWS’s Machine Learning Blog (3 Nov 2026) pins the response on sequence-level knowledge distillation (SeqKD): an API-only teacher writes full tool trajectories, a programmatic scorer keeps only solved runs, and a small student is ordinary-SFT’d on Amazon SageMaker AI.
The thesis is concrete: sustainable multi-step tool agents are less about leaving the frontier in every loop step than about distilling solved trajectories—with the reasoning text kept.
[1]What happened
SeqKD sits at the “teacher text only” end of the distillation spectrum: no teacher weights or logits, no RL stack—so it still works when the teacher is API-only. For an agent, the sequence being copied is not prose but an ordered run of reasoning, tool calls, and observations.
The pipeline uses ShoppingBench (synthetic shopping problems over a ~2.7M-product catalog) as the grader. 750 problems (250 each for product/shop/voucher) are split into 426 train / 324 held-out. Teacher GLM-5.3 on Bedrock (reasoning on, T=1.0) is sampled twice per problem → 850 trajectories; only constraint-satisfying recommendations survive. Solved trajectories expand to per-step chat rows (~2,600 train / 130 val) with loss on the assistant turn only. The student is JumpStart Qwen3.5-4B via serverless LoRA SFT, then served behind a tool-loop realtime endpoint.
[1]How hard is the evidence
Claims are pinned to held-out Absolute Success Rate (ASR), broken out by intent. Base Qwen3.5-4B averages 0.297; the SeqKD student 0.541; greedy teacher 0.540, sampled teacher 0.511, and a medium frontier reference 0.546. The authors flag a few points of single-run LoRA noise, but “essentially all of the teacher’s task quality” is the headline.
The decisive ablation is reasoning: the same trajectories with reasoning stripped leave tool calls only and drop average ASR from 0.541 to 0.415 (~13 points) on every intent—tool-calls-only still beats base, but reasoning between actions is what carries the student to the teacher band. The student can edge the sampled teacher because rejection sampling imitates a best-of solved slice, not average attempts. The regression check is IFEval: prompt-strict 0.819→0.741 and instruction-strict 0.872→0.813 (~6–8 points), written as the expected cost of specialization rather than hidden.
[1]Steelman and gaps
Steelmans: when the teacher is API-only, SeqKD plus a programmatic grader is one of the few portable distillation paths; T=1.0 sampling then filtering turns an imperfect teacher into a cleaner set; problem-level (not row-level) splits avoid near-duplicate leakage; compressing observations to fit length caps is honest engineering.
Gaps: ShoppingBench is synthetic with hidden gold—transfer to noisy live catalogs, inventory dynamics, and fraud constraints is unproven. The medium frontier comparator is unnamed. Matching task ASR is not matching general ability—the IFEval drop is explicit. The authors themselves frame SeqKD as a strong first step: the hard tail may need on-policy methods (not covered) so the student learns from its own rollouts. Single-run LoRA variance also limits how tightly “match” should be read.
[1]Who pays, and what checklist changes
For product and finance: if agent spend is eaten by per-step frontier calls, the checklist should ask whether success is programmatically gradable, whether teacher trajectories can be rejection-sampled, whether reasoning enters the training target, and whether held-outs are split by problem. For platforms: serverless SFT plus reusable trajectory assets cut the fixed cost of rebuilding a distillation stack per student. For evaluation: gains vs base alone are not enough—report greedy/sampled teacher and a mid-frontier peer, plus one general regression metric.
[1]What to watch in six months
Falsifiable next steps: in non-shopping tool domains without synthetic gold, does reasoning-keeping SeqKD still lift ≤8B students to an API teacher’s greedy band; if the grader is noisy or only human-spot-checked, how much of the rejection-sampling dividend survives; can on-policy follow-ups eat the hard tail without further IFEval damage. The claim is not that small models generally replace frontier agents—it is that for multi-step tool tasks with hard success criteria, SeqKD on solved trajectories (reasoning kept) may be closer to an operable cost curve than leaving the frontier in every loop step.
[1]