shonumi
CGFX Load Order/Display frame issue
Hello, love the emulator overall, the CGFX are a very cool idea. Sorry to bother, but I did run into an issue with the CGFX; I was working with Donkey Kong Land (I tried with both 1 and 3) and when replacing the animations, I learned that the original game shifts the tile layouts dynamically to account for different sprite sizes. The problem I ran into is that when the tiles are rearranged, the CGFX are updated first and GBE+ pushes out a frame before moving the tiles to their proper positions that looks garbled as a result, whereas in the base game the update to the tile positions and the updating of the graphics are meant to be done on the same frame (IE, the updated "new" tile graphics from CGFX are shown to the user one frame early!)
I'd love to be able to colorize these games as most of the graphics are taken from the SNES Donkey Kong Country games and scaled down, so it would be great if it could be fixed. I'm not sure when in the process your CGFX routine is updated but I think there may be a priority issue, and maybe a tick box in the options to delay the updating of CGFX by a frame might help for games that use this sort of dynamic tiling (I believe Rare's other Game Boy games are programmed this way as well, so it's definitely not an isolated issue).
For a good animation to see it in action, this was how the emulator attempted to display the ducking animation from DK Land 1:
Thanks in advance!
EDIT: Here is how it looks when displayed with CGFX turned off. The game does not draw the second frame, but if CGFX are turned on it draws it despite the game itself staying the same.
EDIT2: Sorry, I forgot. Here is the manifest and the relevant frames; go frame by frame (it looks fine with just these two but in larger animations it's very obvious/bad, check the frame directly after Duck3) DKL1 Ducking.zip
Unfortunately, CGFX is essentially deprecated in GBE+. I'm no longer working on it and I don't intend to revisit it in the foreseeable future. While it is present in 1.5, it'll probably be removed going forward into 1.6.
That's unfortunate, it's likely the main/only reason I'd use this over any other emulator (especially for GB/GBC which have a lot of other options). I'd hope you reconsider before dropping one of the biggest things setting GBE+ apart; as neat as the peripheral support is, it's very situational and even with a few quirks the CGFX system feels like something unique that no other GB emulator attempted, but I also understand if it's just not fun to work with anymore.
While I appreciate your support for GBE+ and CGFX, phasing out this feature has been something I've considered for at least 2 years now. There are a few technical as well as personal reasons behind it. I apologize in advance for the long response. I just think that if I'm going to remove such a large part of GBE+, users have a right to know my honest thoughts and motivations.
The biggest concern is maintainability. Most of the work that goes into CGFX isn't so much done inside the DMG/GBC core, but the Qt GUI. Quite frankly, it's really a mess code-wise. Initially things were written just to be functional without an eye towards making it manageable down the road. This is largely a result of not knowing what sorts of things CGFX would need in terms of UI when I first started porting the original GBE's implementation over to GBE+ and adding Qt into the mix. It quickly ballooned, and as a result, there's about 3.5K lines of code that's become difficult to control without rewriting the whole thing.
It's a very good thing that little changed in GBE+ in order to support Qt5 (having originally been written for Qt4), but the same might not apply to future versions of Qt. Being tied to Qt for the GUI is something I plan to address separately in the future, but with the massive amount of code invested into CGFX, it's the biggest and most immediate liability for any Qt incompatibilities that might arise.
CGFX (particularly for GBC games) has proven that a few additional options beyond EXT_VRAM_ADDR and EXT_AUTO_BRIGHT are needed to cover a decent amount of issues. The original idea behind CGFX was to dump 1 tile, edit it, and draw it when running the game. But, in order to successfully work around the way the Game Boy handles graphics, users need an array of options. To me, this is in a way feature-creep. While it would allow people to do more with CGFX, the number of possible additional options I've heard from others and considered on my own come pretty close to basically hacking game itself through the emulator. These invariably complicate the way GBE+ hashes graphics and processes hashes, which is already rather difficult to maintain (thanks to the last changes affecting EXT_AUTO_BRIGHT and so-called partial hash matches).
Besides maintainability, adding all of these new options makes things more challenging for users. A lot of the questions I've encountered about CGFX come from users not fully understanding how each works. For some games, you really need to know what the Game Boy is doing in addition to how GBE+ functions with CGFX. Not a lot of people have that insight. Over the past few years, I've been very reluctant to exacerbate that situation, hence nothing really new has been added to CGFX for quite some time.
On a personal level, adding more elaborate options to workaround a number of uncommon or corner cases doesn't sit well with what I first envisioned years ago (something really analogous to texture dumping/replacement for 3D consoles). Unfortunately, the Game Boy works one way and users expect their own graphics to work another way, and reconciling these two sides is far more involved than I anticipated. While it technically is possible to solve almost anything that doesn't currently work, I'm not too thrilled about actually implementing that. Even if made CGFX near perfect for every use, I don't want to create a system that's overbearing and convoluted, either on the surface (for users) or in the emulator itself (the code I have to deal with).
Lastly, there's simply the lack of interest. I just can't find the same enthusiasm I had years ago. Back in 2013, when I'd first got CGFX working for Tetris, I was pretty excited to break new ground. Over time, however, CGFX has simply lost its appeal to me. My current focus is preserving video game hardware and software through emulation. To me, this goal is far more important than making enhancements. Consider that if in 10, 20, or 30 years from now I change my mind, I can easily jump right back into CGFX. The games themselves and our current knowledge of how the Game Boy functions on a low-level aren't going anywhere. On the other hand, there's a great deal of video game hardware that needs to be researched and documented before it's too late.
One of the earliest devices I looked at, the Barcode Boy, is apparently at an age where the hardware is dying. Capacitors fail, stuff leaks, and PCBs corrode. My own Barcode Boy stopped working, and I've met with two other folks in the past week who reported theirs are similarly malfunctioning. Thankfully the Barcode Boy was reverse-engineered, but there are plenty of other cases with even older hardware that no one's looked into yet. These things are time-bombs that don't exactly wait for people. One day, they will all deteriorate for good, so that's where my sense of urgency and priority come from. So, put another way, I could conceivably work on CGFX any year/month/week/day of my choosing. For all the odds and ends of video game history, however, there's only a finite amount of time I have to study them.
Thank you for explaining why you decided to remove support for cgfx. I was shocked to hear it, but I realize now that there are more important things that need to be done. Good luck with your work :)
In a sense I understand the desire to make the process simple to use for the end user and the difficulties involved with keeping that simplicity, but I also feel like a dedicated user of the program who is able to work around those complexities is exactly the type of person a feature like CGFX is for; redoing graphics for a game is a big undertaking (even for a smaller-scale game!) and requires dedication and a desire to see a project to completion; it DOES require a little bit of technical knowhow.
Speaking for myself mainly here, I'm no stranger to hacking Game Boy games; in the past few days even I've found a software-side ROM hack I can do to DK Land 3's animation system to work with the existing CGFX engine and potentially still complete the project (though sadly DK Land 1 is programmed in a way that would make it inconvenient, c'est la vie). I definitely feel like abandoning the feature when the GB/GBC branch of the emulator is overall otherwise functional may be a bit misguided, and shifting the focus on keeping the UI basic to telling the user straight-up "CGFX is meant for those with at least a basic understanding of GB/GBC architecture" might honestly be better for the project and the feature. That said, if the problem is maintaining compatibility with future versions, it does seem like the GB/GBC core in the project is not being actively focused on, and rather than deprecating the feature to maintain compatibility on future cores, it might be better to fork the GB/GBC branch instead of abandoning something that's potentially a big draw to the project.
I also should say just in case you or someone else more familiar with C++ than me wanted to look into it, I did look into the issue briefly while attempting to hack the game's animation engine, and I believe the problem is that the games write directly to VRAM instead of switching which tiles OAM points to, which is likely to be the case with more graphics-intensive games that need to update a lot of animations quickly; I believe your program does not recheck the hash when VRAM is written to directly (it checks the OAM and updates based on that), so I guess if you did decide against scrapping it eventually, that would be what to look into I think; it potentially would fix a lot of games even outside of the edge cases I've been working with.
I do hope you'll rethink abandoning such an interesting feature, but I fully understand if you want to focus on something else. I wouldn't have known your emulator existed if I hadn't been searching for alternative options for coloring GB games, and a friend I was talking to about it similarly knew GBE+ as The Game Boy Emulator With The Texture Pack System and I feel like people looking for a similar project will find there is no other emulator doing what yours does. But I do wish you luck with working on the other features and I hope that the emulator does well. I know it's a lot of hard work and it's mainly a hobby, so I wouldn't want to add extra pressure if the CGFX engine is just too cumbersome to deal with anymore.
GBE+ is open source, so thankfully the CGFX code will still be available even when I remove it. I suppose this is an opportunity for someone else to pick up where I leave off. I really don't mind if others fork it or make something new altogether. Making this emulator has never been about trying to draw attention or become popular (originally, I just wanted to make something that fixed my annoyances with VBA-M). So for any improvements and further work, I'm simply going to leave that to others more interested and more willing than I am.
Anyway, to finally address some of the technical aspects of this issue, you are correct about the direct VRAM writes. GBE+ does monitor all writes to VRAM to update CGFX, but only for background tiles.
The solution would be to check OAM during VRAM writes and see if that memory falls within the OBJ's 8x8 or 8x16 tiles and signal an update. There's not really a clean way to do this at the moment, since CGFX does not keep a list of OBJs that need to be hashed unlike BG tiles. The quick and dirty way would be to force GBE+ to update that specific OAM entry whenever its VRAM data changes, since the OAM update code performs the hashing. The proper way, obviously would be to just use a list of OBJs that need hash updates.
Struggling to get the thing to compile on Windows to even try and experiment with it; it's definitely a bit frustrating to fully understand the programming error and not be able to just go in and fix it myself, but C++ sure is not my forte... I normally program in Python, so all the cross-console SDL2 stuff is alien to me. It doesn't seem like Windows cmake will properly read find_SDL2s and even setting the path manually in the cmake settings doesn't seem to do anything, so I guess I'm unfortunately at an impasse with this even after hours of trying to just get it to compile directly without making edits to the source. I hope someone eventually comes along and fixes it, but I guess I'll just need to abandon the project for now and hold onto the files until someone fixes the thing up. It's a shame though, I got a good amount of the tilesets of the first game ported over and it looked quite good, so it feels bad to scrap it...
I have created a fork and added VS project to it. Currently only compiles to x86 with QT5. Instead of a symbolic path, I put the actual path to the QT5 and SDL2 folders required, so you need to change them to your own location. This should help those who wants to compile the program but don't know how to work with cmake.
