Code2Video-AA: How Models Design Videos in Code — and What It Costs

Artificial Analysis published Code2Video-AA on October 26, 2026. It is AA’s independent implementation of HeyGen’s Code2Video Bench: a set of design briefs for short product and brand videos where each model designs the clip in code. That framing matters. If you are shopping for a model to generate a photoreal shot the way Kling or Sora do, this is the wrong board. If you are choosing a model to write HTML and GSAP for UI demos, kinetic type, or product explainers, Code2Video-AA is much closer to the job. Related reading on HeyGen’s wider video stack: HeyGen Video Goes Beyond Avatars and Generates the Whole Scene.

Figures below follow Artificial Analysis’s 26 October 2026 launch article (dataset 0.3.0). AA also keeps a live leaderboard that can refresh later; if the board moves, treat this piece as a snapshot of that launch day—not a forever ranking.

What “design in code” actually means here

The pipeline is deliberately narrow:

1. AA sends the model a design brief plus HeyGen’s system prompt.
2. The call is single-shot and tool-free—no browsing, no multi-step agent loop.
3. The model returns one HyperFrames composition: an index.html that uses HTML for layout and GSAP for animation.
4. AA renders that composition into a video file.
5. A Gemini 4 Argon judge runs pairwise Elo on five axes; the published score is the mean of those five Elos, anchored so GPT-5.5 (xhigh) = 1000.

So the model is graded on whether the code-designed short looks and behaves like a competent motion designer’s pass—not on whether it can invent camera moves inside a diffusion video generator. Public-split briefs in the article include styles such as a locked screen-recording wireframe demo, kinetic type into a product reveal, and chat-bubble color cycles. That is product-marketing motion language, not open-world cinema.

Methodology lives on AA’s intelligence-benchmarking page under Code2Video-AA: brief selection, generate/render, judging, comparison sampling, and Elo fit. This piece summarizes the launch article and that methodology; the bench has not been re-run on this site.

Launch leaderboard: who is ahead

Scale at launch: 27 models × 114 briefs. Key Elo and AA task-cost figures from the article:

Opus’s lead is 93 Elo over the next model. AA notes the lead is largest on craft (128 points over Sonnet) and smallest on prompt intent (48 points). Temporal consistency separates models the least (Elo range 1057 vs 1222–1349 on the other four axes). Model order is almost the same on every axis—no model moves more than six places between an axis and the overall Elo—so the mean is not hiding a wild axis swap.

In practice: if your demos fail on polish (spacing, type hierarchy, easing), craft is the axis you care about most; if models ignore hard constraints in the brief (locked camera, exact palette, “no zoom”), prompt intent is the failure mode to watch. Temporal consistency moving models the least suggests that, on this cut, “does the motion hold together over a few seconds” is less of a separator than “does the design look finished and on-brief.”

GPT-6.1 Sol (max) at 1369 / ~$0.31 is the cost story most product teams will screenshot: fourth overall at about a tenth of Opus’s ~$3.08 AA task cost. GPT-6 Astra (max) sits near Sonnet on Elo but at ~$1.62—about 5× Sol’s task cost for 58 more Elo points. That is a clean “is the extra craft worth 5× on this evaluation?” question for anyone budgeting demo loops. If you already batch dozens of HTML motion variants a day, Sol’s fourth-place Elo at a tenth of Opus’s AA task cost is usually the first spreadsheet row; Opus becomes the escalation path when a launch film or sales-critical demo needs the top craft score and you can absorb ~$3 per evaluated generation in AA’s accounting.

Cost spread, Pareto points, and open weights

Across all 27 models, AA evaluation cost per task runs about $0.01–$3.89—more than 300×. Eight models sit on an Elo-vs-cost Pareto frontier, from cheap open-weights points up to Opus at 1536 / $3.08. Bookmarkable frontier points in the article include Haiku 5.5 at 1192 / $0.08 and Sol at 1369 / $0.31.

Open-weights trail by at least 483 Elo. The top open-weights names on this cut are DeepSeek V4.1 Flash (max) at 1053, and MiMo-V2.6-Flash / MiMo-V2.6-Pro around 1052. Treat that as a floor check if downloadable weights are a hard requirement—not as a claim they match Opus on craft for the same briefs.

All dollar figures above are Artificial Analysis evaluation cost per task (split by input, cache, reasoning, and answer token pricing where AA reports it). They are not a quote for a finished client film, agency hours, stock footage, music, or voiceover. If a vendor later quotes you a “per video” creative package, that number will include people, revisions, and delivery—Code2Video-AA’s $/task is only the model-call slice AA measured under its fixed brief and single-call rule.

Render rate and why token shape matters

A composition must render to video before the judge can score it. At launch, 3042 of 3078 tasks rendered (98.8%). Eight of 27 models cleared 114/114. The lowest render rate was GLM-5.3 (max) at 110/114. A failed render loses every comparison against a video that did render—so Elo is not “taste alone.”

Output tokens per task span about ~11k–176k; most of that variation is reasoning tokens. Final answers that hold the compositions sit about ~4k–12k across models. The three Claude 5.5 variants use the most output (~152k–176k per task) against a mean around ~42k and a median around ~34k for the other 24 models. That pattern lines up with both higher Elo and higher AA task cost for the Claude 5.5 family—and with Sol’s interesting fourth-place Elo at a much lower task bill.

How to use Code2Video-AA when you pick a model

Separate three jobs that people often mash together:

1. Pixel video generation (Kling, Sora-class, and similar): invent frames from a prompt. Not what Code2Video-AA ranks.
2. Video understanding (timeline Q&A, retrieval over a clip): DigIntoAI’s Gemini mode guide is a better entry—How to Choose Gemini Video Mode—Static 1 FPS or Agentic Retrieval.
3. Code-designed short motion (HTML/GSAP product demos, kinetic type, UI explainers): this board.

Practical cuts from the launch numbers:

• Maximize craft on this evaluation → Opus 5.5’s 1536 and clean five-axis lead.
• Strong design with a much smaller AA task bill → Sol fourth at ~$0.31, or Haiku’s Pareto point at ~$0.08 / 1192.
• Paying up within the GPT-6 family → Astra’s ~$1.62 buys ~58 Elo over Sol on this cut; decide if that craft gap matters for your demos.
• Must stay on open weights → expect a ≥483 Elo gap; inspect render rate and run your own HTML/GSAP QA.
• Building product-site demos or subtitle motion → map your pipeline to brief → HTML → render. Even if you never use AA’s judge, the failure modes (broken GSAP, bad layout, render crashes) are the ones this bench stresses.

A workable team workflow on top of those cuts: keep a cheap frontier model (Haiku or Sol on this board) as the default for draft passes; promote only the variants that survive human design review to a higher-Elo model for the final HTML; then render and QA in your own player before anything ships to marketing. That keeps AA’s cost-per-task spread from becoming your entire production budget, while still using the leaderboard as a relative craft signal.

Two more caveats worth keeping next to the table. First, the judge is Gemini 4 Argon—a strong model from the same wider ecosystem that also appears as a *competitor row* on the board (Gemini 4 Argon high sits mid-pack on the launch Elo list). That does not invalidate the pairwise setup, but it means you should read Elo as “how Gemini 4 Argon ranks these rendered shorts,” not as a human creative-director panel. Second, the briefs are product/brand design tasks under CC BY 4.0 from HeyGen’s public split; your internal brand kit, legal review, and localization still sit outside the score.

For another Artificial Analysis board DigIntoAI has covered, see Ant Group's Ming-3.0-tts Ranks Near the Top on Artificial Analysis Open TTS Boards.

Chinese and Japanese text layout and real production results on this bench have not been tested on this site; feedback can go through the site-owner contact link in the footer.