generative-models icon indicating copy to clipboard operation
generative-models copied to clipboard

Fix MP4 video output in `save_video_as_grid_and_mp4`

Open FlyingFathead opened this issue 1 year ago • 3 comments

Summary:

This PR addresses a critical issue in the save_video_as_grid_and_mp4 function when running video_sampling.py, where video files intended to be saved as MP4 were being mishandled, leading to crashes when writing to MP4 files. The function has been updated to correctly leverage imageio and ffmpeg for MP4 video creation.

Changes:

  1. FFmpeg Integration:

    • Added a system-level check for FFmpeg availability using shutil.which("ffmpeg"). If FFmpeg is not installed, a clear runtime error is raised, prompting the user to install it.
    • Incorporated a check to ensure that the imageio-ffmpeg package is properly installed and functioning in order for the MP4 video encoding to proceed without issues.
  2. Video Encoding with Imageio:

    • Corrected the writer format in save_video_as_grid_and_mp4 by ensuring the use of the ffmpeg format and libx264 codec for MP4 output.
    • Implemented a confirmation message to notify the user that the video has been successfully saved to the specified path.
  3. Dependency Update:

    • Updated requirements.txt to include imageio[ffmpeg]==2.26.1, ensuring that all required video processing dependencies are installed when setting up the environment.

How to Test:

  • Ensure FFmpeg is installed on your system and accessible via your system’s PATH.
  • Run the application with the updated code. If FFmpeg is not available, a runtime error will prompt you to install it.
  • After video sampling, check if the resulting MP4 is correctly created in the specified folder and playable.

FlyingFathead avatar Sep 07 '24 21:09 FlyingFathead

Fixed the check method on imageio[ffmpeg] avaialbility; previous wasn't applicable after all. Feel free to either include or exclude the checking inside the function, but at the very least the imageio[ffmpeg] pip package should be in the requirements, since it hadn't been included all in any of the install scripts.

FlyingFathead avatar Sep 07 '24 22:09 FlyingFathead

For scripts/sampling/simple_video_sample.py vscode also can not view the mp4 file. after downgrade to imageio[ffmpeg]==2.26.1 it works fine.

haochengxi avatar Oct 08 '24 02:10 haochengxi

@RahulVadisetty91 @xijiu9 I've addressed your comments:

  • Refactored the device variable to use the global declaration consistently.
  • The version I proposed uses imageio[ffmpeg] 2.26.1 as suggested.

Feel free to chime in if you think it needs something more.

FlyingFathead avatar Oct 13 '24 17:10 FlyingFathead