openMittsu icon indicating copy to clipboard operation
openMittsu copied to clipboard

Could not insert media

Open o11y21 opened this issue 4 years ago • 17 comments

when receiving an image it crashed with

[2021-02-08 12:06:11.216] [main] [warning] Can not display MessageBox for Exception Handling: Not in main GUI thread.
[2021-02-08 12:06:11.216] [main] [critical] Caught an exception of type InternalErrorExceptionImpl in the main program loop with cause: /home/olly/src/openMittsu/src/database/internal/ExternalMediaFileStorage.cpp:156: 
Could not insert media data into 'media', item with this UUID already exists, but size ('165581' vs. '56505'), type ('1' vs. '2') or checksum ('3307358402' vs. '2268227232') do not match! 
Query error: UNIQUE constraint failed: media.uid Unable to fetch row
[2021-02-08 12:06:11.224] [main] [critical] Since the worker thread just terminated, everything will close now.

I have started with an new ID Backup without contents. Where are the duplicates from?

o11y21 avatar Feb 08 '21 11:02 o11y21

Could you check for me whether 23e5c59e9b3261562d76a334e79f4c943e22cc37 (master) fixed the issue? I found a bug in the de-duplication handling which could have caused this. Otherwise, please share a bit more of the log messages, and if not already done, activate debug logging (CMake: -DOPENMITTSU_DEBUG=On).

blizzard4591 avatar Feb 08 '21 12:02 blizzard4591

this changes it's behaviour to

[2021-02-08 14:49:52.219] [main] [critical] Caught an exception of type InternalErrorExceptionImpl in the main program loop with cause: /home/olly/src/openMittsu/src/database/internal/ExternalMediaFileStorage.cpp:162: Could not execute media existance query for uuid "d12d0ac3-b552-4117-a147-e54ede59f387" table 'media'. Query error:  
[2021-02-08 14:49:52.229] [main] [critical] Since the worker thread just terminated, everything will close now.

There is nothing after "Query error:"

I will add more debug in a moment

o11y21 avatar Feb 08 '21 13:02 o11y21

Weird, I pushed a new change that should make it behave, even though I still don't totally understand what is wrong. If you know how it works, please check the primary key definitions on media in openmittsu's database. SQLiteBrowser (https://sqlitebrowser.org/) can open it, just make sure to start the version with SQLCipher. The primary key should be on uid and type, not only on uid...

blizzard4591 avatar Feb 08 '21 14:02 blizzard4591

Hmm, strange - copied media to a new media table with PRIMARY KEY(uid,type), but no change

[2021-02-08 15:39:47.351] [main] [critical] Caught an exception of type InternalErrorExceptionImpl in the main program loop with cause: /home/olly/src/openMittsu/src/database/internal/ExternalMediaFileStorage.cpp:162: Could not execute media existance query for uuid "3d2fb1f8-6906-48ae-bf90-1997c15e5bb9" table 'media'. Query error:  

Even with Debug no more details. Sending of images works fine.

o11y21 avatar Feb 08 '21 14:02 o11y21

Is that with my fix in ff1115f7c05f07803afc4d5ea167d63166e6d540? Wait, does that mean the table did not have the correct key?

blizzard4591 avatar Feb 08 '21 14:02 blizzard4591

Wait, does that mean the table did not have the correct key?

yes, it had PRIMARY KEY(uid)

o11y21 avatar Feb 08 '21 14:02 o11y21

Is that with my fix in ff1115f?

starts in an neverending loop of something like this

[2021-02-08 15:52:51.235] [main] [error] Detected UUID duplication when inserting file with UUID 509d6aa2-b4e9-44dc-89e8-f670db87e509, size 56505 and file type 2 (if this message appears multiple times, something major is broken!).

o11y21 avatar Feb 08 '21 14:02 o11y21

The root cause was that for newly created databases, the media table was not created correctly. That this was not discovered earlier is a really, really bad sign... Well, it is fixed now. Opening your database with the new version should automatically fix it (uid is not UNIQUE anymore and the PK is over both columns). Please report back :)

blizzard4591 avatar Feb 08 '21 15:02 blizzard4591

yepp, this works. It changes media table by itself. No more errors in receiving pictures. Only they are not being displayed ;) (sent pictures can be seen in history)

o11y21 avatar Feb 08 '21 16:02 o11y21

That is actually a different problem - Threema most recently switched to sending images as "files" (like, the internal IDs and techniques used for sending were switched), and openMittsu is not (yet) ready to detect images in files, so it only says: heres a file.

blizzard4591 avatar Feb 09 '21 11:02 blizzard4591

I see, it seems threema itself is in progress. Does openMittu benefit of there strategy to distribute client software as open source development?

o11y21 avatar Feb 09 '21 12:02 o11y21

Not really, no. But I implemented some necessary changes, the images should now be displayed :)

blizzard4591 avatar Feb 10 '21 23:02 blizzard4591

Not really, no

I thought also about comparing reverse engineered to disclosed code.

But I implemented some necessary changes, the images should now be displayed :)

That's fine - it is working flawless now. Only in groupchat there is no message about new media at all.

o11y21 avatar Feb 11 '21 10:02 o11y21

Why oh why do you find every bug there is - not that I am complaining, it's good. Yes, Group file messages were not completely implemented and dropped. I fixed this and some other issues around it. Check out the latest push @ 96bcc73676768343819e8d2736398f672b156db9.

blizzard4591 avatar Feb 11 '21 14:02 blizzard4591

Amazing, you are unbelievable fast in fixing - I didn't planned to be bug reporter, but like to use messenger services on desktop. Okay, I will slow down then. It's quiet good as it is.

o11y21 avatar Feb 11 '21 16:02 o11y21

No, please keep reporting, this is what keeps the project alive. It was more of a "either nobody bothers to report this issues, or nobody is actually using it" kind of realization.

blizzard4591 avatar Feb 12 '21 13:02 blizzard4591

I see. At the moment I have more feature requests - like showing replies

quote #6be42b4769eb2cb3

o11y21 avatar Feb 12 '21 15:02 o11y21