BroadWay: Boost Your Text-to-Video Generation Model in a Training-free Way

Jiazi Bu1,4*    Pengyang Ling2.4*    Pan Zhang4†    Tong Wu3    Xiaoyi Dong4    Yuhang Zang4    Yuhang Cao4    Dahua Lin3,4    Jiaqi Wang4†   
1Shanghai Jiao Tong University    2University of Science and Technology of China    3The Chinese University of Hong Kong    4Shanghai Artificial Intelligence Laboratory   
* Equal Contribution      † Corresponding Authors;
Paper Github Cite
Demo Video

BroadWay provides a training-free and plug-and-play option to enhance the overall quality of current T2V backbones with negligible additional cost.

Method Overview

BroadWay is composed of two principal components: (1) Temporal Self-Guidance improves the structural plausibility and temporal consistency of generated videos by reducing the disparity between the temporal attention maps across various decoder blocks. (2) Fourier-based Motion Enhancement enhances the magnitude and richness of motion by amplifying the energy of the map.


AnimateDiff Video Gallery
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
Vanilla result generated by AnimateDiff
Result generated by AnimateDiff + BroadWay
VideoCrafter2 Video Gallery
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Vanilla result generated by VideoCrafter2
Result generated by VideoCrafter2 + BroadWay
Image-to-video generation

BroadWay also exhibits potential in the image-to-video (I2V) domain, further expanding its applicability across various video generation tasks.