mediapy load, visualize or write a video with audio

[yangyi02]

Hi, would like to ask if mediapy can load, visualize or write a video with audio in a colab notebook.

I checked the functions in mediapy, but cannot find any audio related code.

I wonder if adding audio into the function is possible or not?

Thank you.

[hhoppe]

Currently there is no way in mediapy to create a video with audio. It should be feasible because mediapy relies on ffmpeg to create the video container and ffmpeg allows the merging of audio and image streams. (The playback of the video container is done using an HTML <video> element which does support audio.) It's an interesting project that I could look at. One would have to define a separate 1D stream of audio samples, with sampling rate and bitrate different from the image stream. I'm curious what applications there are for this.

Note that, to simply play an existing video container (such as MP4) with audio, this can be done by embedding the data or URL in an HTML <video> element. For example see https://stackoverflow.com/a/60986234/1190077.