OpenVisualCloud
[REQ] Better VIDEO super resolution models
Hi there, just discovered this interesting HW-optimized library for IMAGE super resolution.
As well explained in this demo video for VideoGigaGAN (paper here) by University of Maryland/Adobe Research, there are multiple factors to consider for "correct" VIDEO super resolution inferencing:
https://github.com/videogigagan/videogigagan.github.io/raw/refs/heads/main/assets/videos/demo.mp4
Anyway, since VideoGigaGAN sources aren't available, I recommend you to "port" alternative (more contemporary, if possible) VIDEO processing-oriented algos for better/consistent results... ...here are some interesting projects - you can already exploit - to carefully evaluate:
- StableVSR: Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models (paper);
- FMA-Net: Flow-Guided Dynamic Filtering and Iterative Feature Refinement with Multi-Attention for Joint Video Super-Resolution and Deblurring (paper)
- EvTexture: Event-driven Texture Enhancement for Video Super-Resolution (paper)
- MIA-VSR: Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attention (paper)
- STVSR-NO: Space-Time Video Super-resolution with Neural Operator (paper);
- STINet: Enhancing Space-time Video Super-resolution via Spatial-temporal Feature Interaction (paper);
- GIRNet: Global Spatial-Temporal Information-based Residual ConvLSTM for Video Space-Time Super-Resolution (paper); ...
Resources collections:
- HyMPS \ VIDEO \ AI-based \ Upscalers;
- Awesome Video Enhancement \ Video Super Resolution;
- upcoming CVPR 2025, ICME 2025, ICML 2025 and ICCV 2025 results, of course.
Last but not least, I strongly suggest you to enable the Discussion section for this repository in order to let 3rd-party devs/users to exchange ideas & knowledge to push the evolution.
Hope that inspires.
Thank you @MarcoRavich
To enable DL VSR models, we also have the library and FFmpeg plugin: https://github.com/OpenVisualCloud/iVSR, it's based on OpenVINO. For example, it supports the Enhanced BasicVSR model which was optimized based on the public BasicVSR model.
Just discovered @ArcticHare105's S3Diff (Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors paper implementation) that looks VERY promising too (for image upscale/enhance)...
Hope that inspires !