libretro
[request] finding thumbnails by filename aswell and not only by game/label name
Hello :) I wish RA would use/find thumbnails with the same name as the file name.
Actual behavior is that RA uses (and gets) only thumbnails with the exact game/label name.
Why ? Example: I added Atomiswave + Naomi roms games for use with Flycast. example romname: dolphin.zip I do a manual scan to make a playlist called "Atomiswave". The result are all games listed and working, but the list only shows the filename as gamename. Because there're no Atomiswave + Naomi thumbnails, so I want to use the ones from MAME. They have the same name as the rom, so for the example: dolphin.png I've put them manually in the associated folder incl. Named_Boxarts/_Snaps/_Titles. So the thumbnails with the same name as the files are shown, very nice. To get a good gamename I rescan with using a newer Mame DAT file. Which works great :) The example gamename is now "Dolphin Blue". But the thumbnails aren't shown anymore :( So RA doesn't use dolphin.png. Instead thumbnail has to be named as the gamename, so for the example: Dolphin Blue.png. That's a pity. It would be aswesome if RA uses the thumbnails found the old way + search as alternative for a thumbnail with the same name as the game file. On the other hand: Why isn't RA getting the thumbnails from MAME ? If I do a manual scan using a MAME DAT file, RA is finding the game names but not the thumbnails ? Why not ?
in short
actual behavior (example): filename: dolphin.zip gamename: Dolphin Blue thumbnail has to be named: Dolphin Blue.png
wanted behavior (example): filename: dolphin.zip gamename: Dolphin Blue thumbnail can be named: Dolphin Blue.png OR dolphin.png
https://github.com/i30817/libretrofuzz
edit: i'm tired so i'm not going to explain this well, but basically, the MAME names from the dat should be relatively uptodate with the names on the thumbnail servers, and if it's only minor changes, this can help you 'adapt' and find the most similar names. It does not use filenames, just labels though.
And it won't work if the name is not on the servers (which you can fix by contributing to them with a PR). Ironically, for such famous games, The 'Capcom Vs. ...' and the 'Daytona ...' series have no png in the server and the 'Dead or alive' series only has the png for one game, which is almost worse (because the thing 'adapts' to use that png - for dead or alive ++ - to all the games).
Ok, thanks a lot. I'm sure that will help me. 👍 But my main problem wasn't the downloading, I already have the images I want to use. Just the connection between roms and images.
Ok, thanks a lot. I'm sure that will help me. +1 But my main problem wasn't the downloading, I already have the images I want to use. Just the connection between roms and image.
edit: before reading this, read the next post because i slightly screwed up the order of this idea.
you can try some things - the program both downloads and renames to the playlist labels if the names don't exactly match (with some options) but you can edit it to skip downloading if you don't want to download again, and collect the files from a local directory. Install the program, rename or move your thumbnails mame dir to another name / dir, then edit the code of the script (the script will be on windows on the python install dir 'Scripts' subdirectory in a directory libretrofuzz as '__main__.py', or in linux as '/home/i3/.local/lib/python3.8/site-packages/libretrofuzz/__main__.py' where the python version may be different.
Then edit the the code at line 181 from this:
for tdir in ['/Named_Boxarts/', '/Named_Snaps/', '/Named_Titles/']:
lr_thumb = lr_thumbs+tdir
try:
soup = BeautifulSoup(urlopen(lr_thumb, timeout=10), 'html.parser')
l1 = { unquote(Path(node.get('href')).name[:-4]) : lr_thumb+node.get('href') for node in soup.find_all('a') if node.get('href').endswith('.png')}
except HTTPError as err:
l1 = {} #some do not have one or more of these
args.append(l1)
to this:
for tdir in ['/Named_Boxarts/', '/Named_Snaps/', '/Named_Titles/']:
lr_thumb = '/home/i3/Documents/Projects/ScummVM-thumbnails'
lr_thumb = lr_thumb + tdir
l1 = { unquote(node.name[:-4]) : 'file://'+lr_thumb+node.name for node in Path(lr_thumb).glob('./*.png') }
args.append(l1)
where the first 'lr_thumb' string path is pointing to where you moved your thumbnails instead of what is there (strings are contained between quotes, single or double in python if you don't know, and in this case you should end the path without a final directory separator because it will be glued to tdir in the line below that already has it).
Then run it again with libretro-fuzz --no-fail --no-meta (maybe to the last option - it removes the round parenthesis parts of the names from influencing the match, which from your name example you don't have in the filenames), choose MAME and MAME (if the playlist name is MAME.lpl, chose the right name if it's not) and see if some of the files are copied and renamed.
May help you if by 'forcing' a copy+rename of the most similar and using the local files transformed into the 'MAME' playlist label names.
Expect false positives with this strategy if you don't have thumbnails for all the games, or if some are 'more similar' than the others you'd expect. Nothing to be done there but it may help save time for you and you can simply delete those - and reinstall the script - same as install in the readme and try with the files from the server, if they have atomiswave already, by using the script as intended.
It will not redownload files if they already have a match in the local files (so if you delete a single name from the 3 directories by searching for a game name that 'has the wrong thumbnail' from the 'new' retroarch thumbnail dir (not the dir you moved, the one created by the execution where it should be in the RA thumbnails directory), at the dir for that playlist, and delete all of those, then you can use the program as intended.
Then it will attempt to download the remote name that 'most matches' a thumbnail label in the thumbnail server, if there is one. Depending on if you used the '--no-fail' switch or not, it may give up because there isn't a name 'sufficiently similar' to a playlist label for that game without thumbnails. In non-local execution and taking the name from the server, i do not recommend using --no-fail --no-meta because the names of the playlist created by a MAME dat file should match the names on the thumbnail server well so there is no need of removing the parenthesis substring and it will help filter false positives.
The original files are kept (it's better to move the containing directory to not mix the originals and the new copies/renames, that's why i wrote to move it), so you can restore everything if it doesn't work out.
That said, I think you should first try invert (download from the server first then from the local dir) to download normally, by simply rename the MAME (if the playlist is mame again) thumbnail dir for a backup and use the program as intended to redownload everything.
THEN if you have thumbnails that don't exist in the https://thumbnails.libretro.com/MAME/ server (which is kind of normal, the server depends on technical users adding thumbnails), try the scheme i put in on the previous post by editing the script to your local dir to fill the 'holes'.
Inverting the process has the advantage you'll do less deletions at the cost of more download on your internet because there is a much lesser chance of false positives (remember the script edit / invocation i explained in the previous post requires --no-fail which will rename one file to fill - the most similar - even if it's 'obviously' not a similar name, which this version will not in most cases, only the ones where there isn't a thumbnail match in the server).
Thank you very much for the effort. But that's too much for me, I'm just a noob. Sorry 😞
ok, but seriously you can try just to install libretro-fuzz and call it and point it to the playlist and select the platforms that the thumbnail server probably has atomiswave games for instead (dreamscast and MAME probably EDIT: i checked, MAME has them), if you don't mind redownloading. Those posts above were only 'if you want to reuse your personal thumbnails'.
There are only 30 or so, so it's probably less of 50 mb download.
Seeing as most dat files have crc, md5 and sh1 checks why not just do one of those as a fallback?
Game not found? Do md5 check Then download image and rename to games name.
Something like this would make it pretty infallible.
Because people complain about it being slow.
Seriously, just use the manual scanner. Then you get also hacks scanned, with the 'correct' names. Also i suggest using the 'overwrite existing playlist setting' because then you can use the 'refresh playlist' option in 'settings->playlists->manage playlists without caring about name changes.
You could just add the "slow" checks as an advance setting... This desire to a force users to do manual things when you can make it easier to using code I don't get..?
You can do it once and forget about it (well more or less, it could be done better if there was a button to 'refresh all playlists' instead of one for each). As a bonus you can scan for more extensions than what the default may be (because the checksums are missing, like the wad files for dolphin).
And frankly speaking the checksum alternative has way too many limitations. Hacks and translations don't get picked up, or get picked up with the original name if they are softpatches.
Some games actually change their executable or 'rom' (a misnomer for all personal computer platforms but not only, depending on emulator).
It's a nightmare do scan scummvm games the 'recommended' way. My way is still not easy, but at least it works.
Some platforms require specific, user made configuration to work properly so you'd actually want that configuration to be part of the launcher, but doing that sabotages 'crc' identification (dosbox, but not only).
There is always going to be lot of games that get left out in the cold by the checksum approach, and frankly speaking retroarch doesn't do enough with checksums that i care all that much, and what it does is often independent of the scanner (cheevos) or can be worked around (the thumbnails, with my utility), or aren't used at all because people complained or devs realized it would be difficult and error prone (the cheat system loading is still manual).
The only thing that retroarch does for me that still depends on checksums is the explore tab and that never did work much better than just search for a name and exploring the playlists- the historical information is nice, but not fundamental and even on the platforms that are well known like the snes the information is surprisingly incomplete.
