Five 3D pipelines, one QA gate — how we benchmark cinematic output
A single QA scorecard that grades five distinct 3D pipelines against the same rubric so a rendered beat can be routed to the pipeline that produces it best for the least cost. Passed the 2026-06-18 gate at 85 / 85 / 80 / 70 (overall / face / hair / environment). Differentiates against prompt-shop "AI agencies" that can only show one pipeline.
Outcome metrics
- Pipelines benchmarked under one QA rubric
- 5 (Unreal MetaHuman, NVIDIA Cosmos, Isaac Sim, Blender, Studio composite)
- QA gate — 2026-06-18
- overall 85 / face 85 / hair 80 / environment 70 (passed)
- Unreal MetaHuman renders held to the gate
- 167 clips in qa-results/03-unreal-metahuman/
- Rubric categories scored per render
- face fidelity, hair, environment, motion, lighting, audio sync
The problem
"AI-generated cinematic 3D" is a marketing category, not a technology. Beneath the phrase there are at least five distinct production pipelines with radically different strengths, costs, and failure modes. A prompt-shop can show you one beautifully-cherry-picked render from one pipeline; a production studio has to route a shot to the pipeline that produces it best, at cost, and cover its blind spots with the pipeline next to it. That routing decision is only credible if all five pipelines have been benchmarked against the same rubric, on the same day, by the same evaluators.
The five pipelines
Comuvia holds these five in active production:
- Unreal MetaHuman — full-fidelity character work, especially face and micro-expression. The heavyweight for on-screen dialogue.
- NVIDIA Cosmos — world-model-driven environment and camera. Where Unreal is the actor, Cosmos is the set and the camera crew.
- Isaac Sim — physically-grounded scenes, especially where robotics or physically-driven props are on screen. Environment JSONs describe scenes an engine can re-run under different conditions.
- Blender — deterministic hand-crafted work where the render has to match a specific brief exactly. The fallback when generative pipelines cannot converge.
- Studio composite — the final pass that combines outputs from the first four into one deliverable. Where the seams get hidden.
The QA rubric
Every render, whichever pipeline produced it, is scored on six categories on a 0–100 scale:
- Face fidelity — likeness, skin, micro-expression, mouth articulation on lip-sync
- Hair — strand, movement, edge fringing, translucency in raking light
- Environment — scene coherence, camera-relative lighting, background parallax
- Motion — idle motion during silence, weight-shift, gaze, blink cadence
- Lighting — key + fill, rim consistency across cuts, colour temperature
- Audio sync — visemes vs waveform onsets, drift over a 60-second clip
Each category also produces a PASS / WATCH / FAIL verdict. A render can be technically above threshold on all six numeric scores and still FAIL if any one category surfaces a "customer would notice" defect — the rubric is calibrated for viewer detection, not just measurement.
The 2026-06-18 gate
The five pipelines were held to the same rubric on 2026-06-18. Aggregate result:
| Category | Score | Verdict |
|---|---|---|
| Overall | 85 / 100 | PASS |
| Face fidelity | 85 / 100 | PASS |
| Hair | 80 / 100 | PASS |
| Environment | 70 / 100 | WATCH (the flag category) |
| Motion | (per-pipeline; see below) | mixed |
| Lighting | (per-pipeline) | mixed |
| Audio sync | (per-pipeline) | mixed |
Environment carrying the low score is not an accident — it is the category with the widest pipeline-to-pipeline dispersion. Cosmos leads it; Blender is the backstop; Unreal is workable but not the first choice. That dispersion is exactly why we route shots between pipelines instead of picking one and sticking with it.
Per-pipeline notes
Unreal MetaHuman (167 clips at gate) — the workhorse for face-forward dialogue. Face and lip-sync verdicts sit at the top of the range. Environment falls off when the shot has to sell a large exterior — that is where we hand off to Cosmos.
NVIDIA Cosmos (batch at gate) — leads on environment and camera. Face work through Cosmos is not yet at Unreal's bar; we do not use Cosmos to render faces for the daily production pipeline, only for the environment plate that Unreal composites onto.
Isaac Sim (scene JSONs, stub previews) — the pipeline for physically-grounded scenes. Not a first-choice render pipeline for cinematic work yet, but the scene-description JSON is portable enough that a scene fixed in Isaac can be re-run in Blender or fed as prior into Cosmos. Its value is the shared scene-graph, not the raw render.
Blender (poster + preview + render package at gate) — the deterministic fallback. When generative pipelines cannot converge on what the brief actually needs, Blender is what closes the shot. It also plays a QA role — a Blender re-render of the same beat is a sanity check on whether a Cosmos or Unreal output is drifting.
Studio composite — the last-mile pipeline. Alpha-matte compositing, chyron overlay, colour grading. The seams between the four upstream pipelines get hidden here; a shot that fails at Studio usually reveals a defect that would have escaped notice in any single upstream pipeline.
Why this is a differentiator
A prompt-shop "AI agency" typically shows one pipeline in the pitch deck and hopes the client does not ask about the other four. That approach breaks the moment the shot list includes an exterior wide plus a face close-up plus a physically-driven prop. A studio that routes between five pipelines under one QA gate can honour a shot list built by a director who does not know or care which pipeline is producing each beat.
The QA scorecard is the artifact that makes the routing decision auditable. When a shot has been graded on the same rubric across all five pipelines, we can defend the routing decision to the director without hand-waving.
What the case study does not fit
This is not a case for a client that wants "one AI style" applied to a full film. That is a stylistic decision, not a pipeline-routing decision, and the answer there is Blender + a bespoke shader — not the five-pipeline rig.
It is also not a case for a project with a fixed pipeline mandate from the buyer ("we already run Unreal, please match our output"). The routing value evaporates when the pipeline is pre-picked.
Request the full QA scorecard PDF
The 2026-06-18 QA scorecard PDF (all six categories, per-pipeline breakdown, representative renders inline) is held in a private vault. Access is granted on request to prospective retainer clients evaluating Comuvia against other production partners. Use the contact form and mention "3D QA scorecard" in the message.
Related offerings
- Fractional Architecture Advisor — the retainer that operates the five-pipeline rig for a portfolio company or brand.
- Architecture Review Sprint — the front-door entry point for buyers who want to evaluate their own current 3D pipeline against the QA rubric before committing to a retainer.