š¬ Subject-driven T2V that keeps a reference subject's identity even as it shuttles across domainsāreal ā fantasy.
Title: DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
URL:
DomainShuttle reconciles subject fidelity with flexible style adaptation. Three highlights worth your attention.
𧬠Domain-MoT
Video and reference image are processed in two independent branches; the reference branch uses Domain-aware AdaLN, modulated by time plus a domain attribute (real human / object / background / fantasy subject). Text cross-attention is frozen to preserve the base model's language guidance.
š Video-Reference DualRoPE
Reference tokens get a separate RoPE space from video tokens for precise subject-level spatial control. Video starts its temporal index at 1, reference is fixed at 0, and multiple subjects (or multiple images of one subject) are organized via positional offsets.
š Cross-Pair Consistent Loss
Training uses two different reference sets at the same timestep, suppressing overfitting to single-frame redundancy and extracting the subject's intrinsic featuresāindependent of irrelevant visual properties.
Cross-domain subject consistency hits CD-Score 0.861, +18.7% over SOTA (Kling 1.6 is 0.725). A practical win for realāfantasy style transfer.
#
VideoGeneration# #
GenerativeAI#