On this page
TaoMate-HD is a progressive-resolution autoregressive audio-video generation method from Alibaba’s TaoLive AIGC team. It produces speech and high-resolution video together in small chunks: each chunk is first laid out on a coarse video grid, then refined at the output resolution, with audio generated along the same trajectory. The project page and GitHub README mark the first release of inference code and preview LoRA weights on 2026-11-03. The paper *TaoMate-HD: Progressive-Resolution Autoregressive Audio-Video Generation* is on arXiv as 2611.03394, with citation and online dates 2026-11-02. Those two dates are one day apart; neither is a same-day news spike.
Here is what the public materials actually claim: long-form generation versus live streaming, the eight-H20 FPS numbers versus the 24 FPS output spec, how the MiniMax H3 FL2VA base and preview LoRA fit together, and where the MiniMax H3 Community License draws hard lines. It is a technical reading of those sources, not a product review and not a demo reproduction.
What “hour-long” means—and what it does not
The README’s long-form bullet says a sequence can continue with new prompts without resetting stage-local history, and that the paper demonstrates hour-long streams. The project page frames long-form demos as four scenes of one hour each and “60:00 ONE CONTINUOUS GENERATION.” The paper abstract likewise says the model supports hour-long audiovisual generation.
In plain terms, that is a claim about continuing an autoregressive rollout so a single continuous AV sequence can reach about an hour. It is not a promise of real-time livestream throughput, and not a SaaS live product rating. Official demo UI and videos sit on the project page; the reading below uses the paper, README tables, and static page copy.
For a product-side contrast in the same broad “digital human / scene video” space, DigIntoAI’s HeyGen Video Goes Beyond Avatars and Generates the Whole Scene covers a commercial offering. TaoMate-HD here is a research-style progressive streaming AV stack built on MiniMax H3—not that product.
Method in one pass
README summary of the pipeline:
- Three-step progressive generation. Two evaluations at 480p set coarse structure, then one at 768p adds spatial detail. A coupled-noise lift changes the video grid while audio rides the same path.
- Continuous speech and motion. Each denoising stage keeps its own clean audio-video predictions as history for the same stage of the next chunk. An initial video anchor plus two recent AV chunks carry context across prompt changes.
- Optional pipeline acceleration. LOW0, LOW16, HIGH, and decoding can run concurrently across eight GPUs. Serial and pipelined modes share the same inference adapter and prompt format.
Continuation training uses the model’s own progressive rollouts. The wording above follows the public README.
Official numbers on eight H20 GPUs
The README Comparison table evaluates quality on 100 sequences of 1–10 minutes. For TaoMate-HD it also reports timing that spans prepared input to decoded video and audio on eight H20 GPUs:
| Mode | GPUs | FPS (throughput) | CER (%) |
|---|---|---|---|
| TaoMate-HD (serial) | 8 | 4.68 | 19.57 |
| TaoMate-HD (accelerated) | 8 | 8.99 | 21.28 |
The same README states that the 10-second example was run on eight H20 96 GB GPUs in both modes, at 768 × 1376 with 24 FPS video and 32 kHz stereo audio. The paper’s evaluation notes (rank-128 LoRA on self-constructed distillation pairs starting from MiniMax-H3 FL2VA; outputs 768×1376 @ 24 FPS / 32 kHz stereo) line up with that output spec where they overlap.
Under matched BF16/SDPA execution, the README further reports progressive resolution at about 1.60× versus direct 768p generation (including AV decoding), and an optimized deployment about 1.92× over serial Progressive via stage overlap, LOW-stage fusion, KV reuse, selective W8A8, attention kernels, and codec scheduling. Continuation training is said to cut CER from 30.75% to 19.57% (an 11.18 percentage-point drop). Treat those as vendor/paper figures under the setups they describe.
Generation FPS is not playback FPS
Two numbers get mixed up easily:
- Generation / throughput FPS (4.68 serial, 8.99 accelerated) — how fast the official timing path moves from prepared input to decoded AV on eight H20 GPUs in the reported modes.
- Output video frame rate (24 FPS) — the frame rate of the decoded video file in the documented example settings, alongside 32 kHz stereo audio.
A machine that reports ~9 generation FPS is not “playing” or “livestreaming” at 24 FPS. Conversely, a 24 FPS file does not mean the generator was producing twenty-four new frames of content every wall-clock second under every setting.
Do not take 8.99 (or 4.68) and multiply into “one hour of content always finishes in X minutes of wall clock for every resolution, prompt length, and machine.” Timing depends on mode, hardware, codecs, prompt windows, and whether the same prepared-input→decoded-AV span the README uses is what gets measured. The public materials do not hand out a single end-to-end wall-clock for every configuration.
Base model and preview LoRA
Inference, as documented, needs both:
- Base: MiniMax H3 FL2VA, Hugging Face repo
MiniMaxAI/MiniMax-H3, download includemodel_index.jsonandFL2VA/*. - Preview LoRA:
TaoLiveAIGC/TaoMate-HD-preview, filesconfig.json(LoRA rank and alpha) andadapter_model.safetensors.
The HF preview card describes this adapter as the inference LoRA for MiniMax H3 FL2VA, usable for serial and accelerated inference on eight GPUs, developed by the Alibaba TaoLive AIGC Team and powered by MiniMax H3. The README notes that repositories requiring authentication need hf auth login with an account that has access—access rules are whatever Hugging Face shows that day. The paper mentions a rank-128 LoRA; the preview card’s config.json is the place to read the shipped rank/alpha for the preview files.
Command examples in the README (eight visible GPUs, --mode serial or accelerated, --resolution 768x1376, and so on) are quoted as documentation, not as completed local runs.
License layers (English LICENSE is the source of truth)
The TaoMate-HD release follows the MiniMax H3 Community License Agreement (see the repo LICENSE; release/License date August 2, 2026). README also points to NOTICE for attribution and says upstream license terms still apply to use and redistribution of the base model. This section paraphrases the English text for navigation; it is not legal advice, and it does not mean “unrestricted commercial use.”
Who “MiniMax” is in the agreement. In the LICENSE definitions, We / Us / MiniMax means Nanonoble Pte. Ltd.
Territory. “Applicable Territory” means worldwide, excluding the Excluded Territories. Excluded Territories are stated as the European Union, the United Kingdom, the Republic of Korea and the United States of America. Separately, the agreement bars use, reproduction, modification, distribution, or display of the MiniMax H3 Works or their outputs/results outside the Applicable Territory; such use is not authorized by the agreement. Whether any particular reader’s jurisdiction is in or out is for counsel and the live LICENSE text.
Yearly revenue threshold. If “your commercial products and services” (LICENSE wording) generate more than 20 million US dollars (or equivalent) in yearly revenue, a separate, prior written authorization from MiniMax is required via api@minimax.io with subject line “MiniMax H3 licensing - authorization request”. The clause is about yearly revenue of those commercial products and services, not a one-off contract size. The LICENSE does not spell out accounting formulas beyond that wording.
UI credit. For a commercial product or service that uses MiniMax H3 or MiniMax H3 Works, the UI must prominently display “MiniMax H3”. (The LICENSE text places no space between “H3” and “on” in that sentence; the required display string is still “MiniMax H3”.)
How the pieces layer (do not collapse into “the whole zip is freely commercial”):
| Layer | What it covers | License signal in public materials |
|---|---|---|
| MiniMax H3 base weights / MiniMax H3 Works | FL2VA base and related Works | MiniMax H3 Community License (territory, AUP, >$20M auth, UI credit, etc.) |
| TaoMate-HD inference modifications | Adapter / inference changes | NOTICE: Alibaba TaoLive AIGC Team, 2026; release still follows H3 Community License; upstream H3 terms remain |
| Encoder inside MiniMax H3 | Qwen3-VL-32B encoder | Apache 2.0 (separate LICENSE pointer in the H3 materials) |
| Other deps / codecs / libs | PyTorch, Diffusers, vLLM revisions, FlashAttention, etc. | Each upstream license; do not assume they match H3 |
NOTICE also keeps Apache-2.0 notices for upstream model and codec components applicable to those components. The Acceptable Use Policy attached to the Community License lists many prohibited uses (including use outside the Applicable Territory); read the live Exhibit rather than treating this summary as complete.
Out of scope for this reading
- DigIntoAI hour-long timing, 4.68 / 8.99 FPS measurement, or CER scoring.
- Quality judgments from watching the project-page demo videos.
- Treating 8.99 FPS as a universal wall-clock for one hour of output under every setting.
- Calling the stack MIT/Apache “whole package,” or unrestricted commercial use.
- DigIntoAI downloads of MiniMax H3 or TaoMate-HD-preview weights.
For primary materials, start with the TaoMate-HD project page, arXiv 2611.03394, the GitHub README / LICENSE / NOTICE, the TaoMate-HD-preview card, and the MiniMax-H3 base pointer—then re-read the English LICENSE before any commercial plan.