Technical Name Deep Reinforcement Learning for Video Prediction
Project Operator National Chiao Tung University
Project Host 杭學鳴
Our video prediction approach uses sparse motion field which combines deep RL technique to discover the critical sparse positions and their motion vectors. We apply POBMC method for final frame synthesis. Our prediction shows similar PSNR and SSIM score compared with SOTA but much better human vision perceptual LPIPS score.
Scientific Breakthrough
Our model combines video compression POBMC technique and modern DL and RL. Unlike direct generation, our model explicitly models the motion tendencies. Different from dense method which requires artificial constraint, ours preserves the true motions. Besides, our model size is small and we only need fewer data to achieve good generalization.
Industrial Applicability
From industry application perspective, video prediction can be used in any application that needs to predict the future, such as smart autonomous vehicles. Or video prediction can benefit other tasks by predicting the future first such as activities predictions. This method can also be applied on video compression. We only need to transmit a small amount of motion vectors and their location information.
Keyword Video prediction Deep learning Reinforcement learning Sparse motion vector Overlapped blocked motion compensation
  • Contact
other people also saw