element-android
element-android copied to clipboard
element-hq
Sending webm compressed does not change mimetype (but does change format)
Steps to reproduce
- Have video with webm format, mimetype recognized correctly
- Send video to room, select compression (aka don't select "send uncompressed")
- See file get sent...
- Observe the event of sent file and see that mimetype is still set to video/webm, while content is actually mp4
- (For example) many bridges rely on mimetype being correct in order to do whatever function or conversion needed to work on the "other side" fails.
Outcome
What did you expect?
Sent (and compressed by EA) file having the mimetype of the codec it have used with "compression" after compression.
What happened instead?
In pure Matrix-side amazingly things generally seems to work (so my assumption is that things ignore mimetype for some reason). but bridges that does rely on correct mimetypes fails, and more generally it is false information at best.
Excerpt from example event relevant part with wrong mimetype:
{
"type": "m.room.message",
"content": {
"body": "comp.webm",
"info": {
"size": 6410676,
"mimetype": "video/webm",
"duration": 19869,
"thumbnail_info": {
"w": 337,
"h": 600,
"mimetype": "image/jpeg",
"size": 93464
},
Excerpt from Mautrix-whatsapp log from its error on same event:
WRN [matroska,webm @ 0x562a8a33fe00] 0x00 at pos 0 (0x0) invalid as first byte of an EBML number action="handle matrix message" command=ffmpeg event_id=$F021VGv8MFXE9zD6eIxaxeNmoqvopPPQKXhtqHOZVMQ portal_key=<redacted>@s.whatsapp.net-<redacted>@s.whatsapp.net
WRN [matroska,webm @ 0x562a8a33fe00] EBML header parsing failed action="handle matrix message" command=ffmpeg event_id=$F021VGv8MFXE9zD6eIxaxeNmoqvopPPQKXhtqHOZVMQ portal_key=<redacted>@s.whatsapp.net-<redacted>@s.whatsapp.net
WRN /tmp/mautrix_ffmpeg_182907036/input.webm: Invalid data found when processing input action="handle matrix message" command=ffmpeg event_id=$F021VGv8MFXE9zD6eIxaxeNmoqvopPPQKXhtqHOZVMQ portal_key=<redacted>@s.whatsapp.net-<redacted>@s.whatsapp.net
ERR Error converting message $F021VGv8MFXE9zD6eIxaxeNmoqvopPPQKXhtqHOZVMQ from @olmari:hacklab.fi: failed to convert media (video/webm to video/mp4): ffmpeg error: exit status 1 module=Portal/<redacted>@s.whatsapp.net-<redacted>@s.whatsapp.net
Ffprobe results from both compressed and then original file, showing actual codecs etc, with original the mimetype of webm is correct, with compressed it should be mp4 (or to whatever format EA compresses an file):
olmari@morpheus:~$ ffprobe comp.webm
ffprobe version 5.1.4-0+deb12u1 Copyright (c) 2007-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'comp.webm':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2023-11-14T18:26:27.000000Z
com.android.version: 11
Duration: 00:00:19.87, start: 0.000000, bitrate: 2581 kb/s
Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt470bg/bt470bg/smpte170m, progressive), 720x1280, 2003 kb/s, 30 fps, 30 tbr, 90k tbn (default)
Metadata:
creation_time : 2023-11-14T18:26:27.000000Z
handler_name : VideoHandle
vendor_id : [0][0][0][0]
Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 575 kb/s (default)
Metadata:
creation_time : 2023-11-14T18:26:27.000000Z
handler_name : SoundHandle
vendor_id : [0][0][0][0]
olmari@morpheus:~$ ffprobe orig.webm
ffprobe version 5.1.4-0+deb12u1 Copyright (c) 2007-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
configuration: --prefix=/usr --extra-version=0+deb12u1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
libavutil 57. 28.100 / 57. 28.100
libavcodec 59. 37.100 / 59. 37.100
libavformat 59. 27.100 / 59. 27.100
libavdevice 59. 7.100 / 59. 7.100
libavfilter 8. 44.100 / 8. 44.100
libswscale 6. 7.100 / 6. 7.100
libswresample 4. 7.100 / 4. 7.100
libpostproc 56. 6.100 / 56. 6.100
Input #0, matroska,webm, from 'orig.webm':
Metadata:
ENCODER : Lavf58.76.100
Duration: 00:00:19.87, start: -0.007000, bitrate: 171 kb/s
Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv, smpte170m/bt470bg/smpte170m, progressive), 270x336, SAR 1:1 DAR 45:56, 30 fps, 30 tbr, 1k tbn (default)
Metadata:
ENCODER : Lavc58.134.100 libvpx-vp9
DURATION : 00:00:19.840000000
Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
Metadata:
ENCODER : Lavc58.134.100 libopus
DURATION : 00:00:19.869000000
Your phone model
Planet Computer Astro Slide
Operating system version
Android 11
Application version and app store
Element Android 1.6.6
Homeserver
Hacklab.fi Synapse 1.95.1
Will you send logs?
No
Are you willing to provide a PR?
No
