🎬 Subject-driven T2V that keeps a reference subject's identity even as it shuttles across domains—real ⇄ fantasy.
Title: DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
URL:
DomainShuttle reconciles subject fidelity with flexible style adaptation. Three highlights worth your attention.
🧬 Domain-MoT
Video and reference image are processed in two independent branches; the reference branch uses Domain-aware AdaLN, modulated by time plus a domain attribute (real human / object / background / fantasy subject). Text cross-attention is frozen to preserve the base model's language guidance.
📐 Video-Reference DualRoPE
Reference tokens get a separate RoPE space from video tokens for precise subject-level spatial control. Video starts its temporal index at 1, reference is fixed at 0, and multiple subjects (or multiple images of one subject) are organized via positional offsets.
🔗 Cross-Pair Consistent Loss
Training uses two different reference sets at the same timestep, suppressing overfitting to single-frame redundancy and extracting the subject's intrinsic features—independent of irrelevant visual properties.
Cross-domain subject consistency hits CD-Score 0.861, +18.7% over SOTA (Kling 1.6 is 0.725). A practical win for real⇄fantasy style transfer.
#
VideoGeneration# #
GenerativeAI#