MODEL FIELD NOTES · Workflow
Why Keyframe Refinement Causes Face and Action Drift in Reference-to-Video Pipelines
Adding a generative image refinement pass between storyboard generation and video rendering often makes single stills look sharper while breaking character identity and blocking across shots. This article explains why cascade drift happens and how to separate identity anchors from motion control.
阅读本站中文版 · 在黄果制片阅读中文版How full-frame refinement amplifies cross-shot identity drift
In our evaluation of three-stage pipelines (storyboard panel generation, keyframe refinement with Qwen or Seedream edit models, and H3 or Wan video generation), we observed a recurring paradox: each refined keyframe looked cleaner in isolation, yet the concatenated three-shot sequence looked like three different actors.
Full-frame generative refinement rewrites over 70 percent of unmasked pixels, subtly altering jawlines, lighting direction, shoulder angles, and hand contact coordinates on each shot independently. When those three independently redrawn stills are fed into a video model, the temporal sampler compounds the differences rather than reconciling them.
Even when a refinement prompt explicitly instructs the editing endpoint to preserve facial geometry and camera lens perspective, unmasked diffusion passes treat the entire canvas as negotiable latent space. A two-degree shift in eye line or collar silhouette on the refined still often cascades into a completely different head turn once temporal attention layers begin extrapolating motion across five seconds.
| Pipeline Strategy | Unmasked Pixel Rewrite | 3-Shot Identity Continuity | Pose & Blocking Preservation |
|---|---|---|---|
| Direct storyboard panel to I2V | 0% extra redraw | Consistent within panel batch | Preserves original blocking geometry |
| Unmasked full-frame polish to R2V | >70% pixels reconstructed per shot | Low (independent facial drift per shot) | Frequently shifts gaze, hands, and spacing |
| Locked synthetic cast + 3D motion driver | Identity anchor fixed across shots | High (shared cast anchor across s01-s03) | Controlled explicitly via pc_humanoid@1 rig |
Reference-to-Video is a soft hint, not a frame-one lock
Our failure catalogue highlights a second structural trap: treating Reference-to-Video (R2V) as if it locks the reference image as frame one. In R2V mode, the reference acts as a soft identity and wardrobe hint while the model is free to re-compose the opening frame to match the text prompt.
When auditing historical pipeline logs, we also found that some apparent refinement regressions were caused by client schema mismatches in the request payload rather than model weights. Always log the exact serialized request alongside the output digest before blaming a checkpoint.
If your scene requires two actors to begin at an exact interpersonal distance or maintain a specific hand-on-shoulder contact point from the very first frame, passing a polished portrait to an R2V endpoint without structural pose conditioning leaves starting geometry up to chance.
Workflow rules for keeping three shots continuous
Porncut structures short film creation around immutable revisions so that character identity, camera blocking, and take selection do not overwrite one another. Separating who is in the scene from how they move keeps multi-shot cuts coherent.
- Lock the synthetic character anchor once at the ScriptVersion stage instead of running unmasked redraws per shot.
- If a keyframe requires local cleanup, enforce pixel-equality checks outside the inpaint mask.
- Use verified 3D motion drivers and A/B take comparison to fix posture and pacing rather than over-editing reference stills.