AWS Distills GLM-5.3 Into a Qwen3.5-4B Shopping Agent—Near the Teacher, With Tradeoffs

An Amazon SageMaker AI technical how-to on the AWS Machine Learning Blog—by Giuseppe Zappia, Ankita Agarwal, Amin Dashti, and Michael Oguike, dated 02 NOV 2026—walks through sequence-level knowledge distillation (SeqKD) for a shopping agent. The teacher is GLM-5.3 on Amazon Bedrock (reasoning on; trajectory sampling at temperature 1.0). Solved multi-step trajectories from ShoppingBench are kept by rejection sampling, then used as supervised fine-tuning (SFT) targets for a Qwen3.5-4B student via SageMaker AI model customization (LoRA; JumpStart ID huggingface-vlm-qwen3-5-4b). On the 324-problem held-out split, the distilled student’s average Absolute Success Rate (ASR) is 0.541—matching the greedy teacher’s 0.540 within the post’s own “few points of run-to-run variation” caveat, and far above the untuned 4B base at 0.297. That is a shopping-task result from a single LoRA run, not a claim that a 4B model comprehensively matches GLM-5.3. DigIntoAI has not re-run ShoppingBench, not measured endpoint latency or spend, and not verified the IFEval deltas below. Every score in this article is an AWS Machine Learning Blog self-report.

Related reading: AWS Opens Infinity, a Research Runtime for Extreme Agent Concurrency, Deep Principle Opens Hydrogen-1.0 on GLM-5.3, Open Weights Still Need a Runtime, Same Model, Different Coding Harness: Pass Rate Barely Moves, the Bill Does, and Cohere Says Qwen3.8-Flash-Next Tried to Upload to a Beijing OSS Bucket in Testing. Infinity is a neighboring AWS agent-runtime note; Hydrogen is a separate GLM-5.3 weights story; the harness piece is about coding wrappers and bills, not SeqKD; the Cohere note is a Qwen-family availability/security probe. None of them replaces reading this distillation cookbook on its own terms.

Why distill a shopping agent at all

An agentic shopping turn is a loop—search, inspect, refine, verify, recommend—not a single frontier call. Each step bills the teacher. The AWS write-up frames distillation as a way to keep much of that task quality at a small model’s hosting cost and speed: the teacher writes the behavior once as text trajectories; the student learns to emit the next reasoning-and-tool-call under ordinary SFT. SeqKD needs only teacher text—no teacher logits, no weight access—so it still works when the teacher is API-only on Bedrock. Compared with logit distillation or on-policy rollouts, it sits on the portable, first-stage end of the spectrum. The post itself positions SeqKD as a strong warm start that often plateaus on the hardest recovery cases—on-policy methods are named as the natural next stage, not covered in the tutorial.

Pipeline facts (as the tutorial states them)

ShoppingBench supplies a programmatic scorer. That scorer is what turns a capable-but-imperfect teacher into a clean imitation set: failed runs are dropped, solved runs stay. The blog notes the catalog behind the agent is on the order of ~2.7 million products. DigIntoAI has not stood up that stack.

ASR on 324 held-out problems (AWS table, single LoRA run)

Absolute Success Rate is the share of held-out problems the agent solves under the same unmodified ShoppingBench agent loop, averaged across the three intents; missing trajectories count as failures. A problem counts as solved when recommended products meet every stated constraint.

Reading the table the way the blog does: the 4B student sits next to the greedy teacher and an unnamed medium-sized frontier ceiling, not “above GLM everywhere.” Sampling at T=1.0 was used to generate diverse training trajectories; greedy decoding is the stronger one-shot evaluation setting, which is why greedy teacher ASR (0.540) beats sampled teacher ASR (0.511). Rejection sampling trains the student on a best-of slice of solved teacher runs, which is also why a student can sit slightly above the sampled teacher it learned from without contradicting the method. The post explicitly warns to expect a few points of run-to-run variation. DigIntoAI has not retested any cell.

Reasoning in the targets is not optional decoration

Training the same student on the same trajectories with reasoning stripped—tool calls only—drops average ASR from 0.541 to 0.415 (about 13 points). Both students still clear the base model, so tool-call imitation alone helps; keeping teacher reasoning is what carries the student the rest of the way toward the teacher on this shopping loop. For deployment readers, that is a data-design choice as much as a modeling choice: compressing trajectories to final answers or bare tool JSON throws away the behavior the ASR table attributes to the full student.

IFEval: the specialization bill

ASR measures the distilled skill. IFEval is the regression check the blog uses for general instruction following. Prompt-strict falls 0.819 → 0.741; instruction-strict falls 0.872 → 0.813—roughly 6–8 points. The tutorial calls that the expected cost of specialization, not proof the student got uniformly stronger. A much larger drop would be the warning that distillation narrowed the model too far. Again: AWS figures, not a DigIntoAI retest.

Serving shape: adapter on a real-time endpoint

Fine-tuning teaches the student to emit the next tool call as text. Something outside the model still has to parse that call, run the tool, append the observation, and prompt again until a final recommendation. The tutorial treats that loop as ordinary application code—or an off-the-shelf serving harness—not as reinforcement learning.

The student artifact in the walkthrough is a LoRA adapter over an open-weight base. Rather than merging and hosting a second full copy of the weights, the blog attaches the adapter as a second inference component on one real-time endpoint that already serves the base, with the DJL LMI container (vLLM engine) pinned to a concrete version for reproducible redeploys. Naming the inference component at invoke time selects base versus distilled student for side-by-side checks. Instance preference in the sample path is ordered among ml.g5.2xlarge, ml.g6.2xlarge, and ml.g6e.2xlarge. DigIntoAI has not deployed that endpoint or measured its concurrency curve.

A low-cost pre-check also appears before the full ShoppingBench agent loop: SageMaker AI model evaluation with a custom scorer that grades tool selection (right tools in order) against the teacher’s reference action, without requiring exact argument formatting. The decision rule in the post is blunt—the student must beat the base on that gate before the expensive end-to-end run is worth the compute. That gate is part of the cookbook’s evaluation hygiene; it is not a DigIntoAI reproduction.

Cost and latency: shape without a headline dollar figure

The reason to distill, in the blog’s framing, is quality at the small model’s cost shape. A frontier API teacher bills per token on every step of every task; the distilled student runs on a hosted endpoint billed by the instance-hour, so added traffic does not multiply the same way until scale-out. The post is explicit that it does not publish a per-task cost or latency number, because amortized cost depends on traffic pattern, sequence lengths, and concurrency on G-family instances (examples in the post include ml.g5.2xlarge / ml.g6.2xlarge / ml.g6e.2xlarge). Inventing a DigIntoAI dollar-per-task or millisecond figure would be fabrication. The operational takeaway that survives without that number: measure own workload against SageMaker AI instance pricing versus frontier token pricing before treating “near teacher ASR” as an automatic ROI win.

Limits the tutorial itself flags

Capacity wall. Distillation transfers a distribution the student can already represent. Across too large a gap, traces lengthen while accuracy stays flat—well-formed tool calls that make no progress. The gap that decides this, the post stresses, is between task and student, not student versus teacher headline size. Here the teacher is described as a mixture-of-experts with roughly 40B active parameters versus a 4B student—about ten times—yet the shopping task is treated as within 4B reach, which is why recovery of teacher task quality is presented as plausible rather than magical.

Where SeqKD stops helping. Easy and mid-difficulty problems lift; the hardest tail plateaus when the student must recover from its own mistakes in a multi-step loop. That is why SeqKD is framed as a first stage, often followed by on-policy methods that score the student’s own rollouts—outside this cookbook.

Honest evaluation. Measure on held-out problems; compare against more than the untrained base the student is bound to beat—set the student beside the teacher and an independent frontier reference so the number shows distance to what was distilled from.

Operator checklist (we haven't retested)

1. Attribution. AWS Machine Learning Blog technical how-to, 02 NOV 2026; authors as listed above—not a DigIntoAI lab report.
2. Method. SeqKD = teacher text trajectories → SFT; example student train is LoRA on Qwen3.5-4B.
3. Split. ShoppingBench 426 / 324; intents product / shop / voucher.
4. Headline ASR. Student 0.541 vs greedy teacher 0.540 vs base 0.297—single run, shopping only.
5. Ablation. Tool-only 0.415; keep reasoning in targets.
6. Regression. IFEval ≈ 6–8 point drop.
7. Cost. No official per-task figure; endpoint-hour vs token-step is the shape to measure locally.
8. Ceiling label. “Medium-sized frontier model” stays unnamed in the source—no invented competitor ID.
9. Self-report stamp. We haven't retested.

Bottom line

On 2026-11-02, the AWS Machine Learning Blog showed how to distill a Bedrock GLM-5.3 shopping agent into a Qwen3.5-4B student with SeqKD on SageMaker AI: rejection-sampled ShoppingBench trajectories, LoRA SFT, then a real-time endpoint that still needs an ordinary tool harness around the model. In the reported single-run table, average ASR moves from base 0.297 to distilled 0.541, sitting next to greedy teacher 0.540 on that shopping held-out set—while IFEval pays a mid-single-digit specialization tax and per-task dollars stay unpublished. That is a task-local, AWS-reported efficiency story with clear tradeoffs, not a license to say a 4B model comprehensively caught GLM-5.3. Figures above are blog self-reports; we haven't retested. Field notes can go through the site-owner contact in the footer.