libretro
Waifu2x ported as a shader
I was told by a forum member at ngemu to try and ask here. I wonder if anyone can port the Waifu2x upscaler and denoiser as a shader to use for PS1 emulation. You can see the result of Waifu in action here, and it looks amazing. FFIX- http://i.imgur.com/zR1kjlE.jpg
FFVII http://i.imgur.com/CzRczDb.jpg
FFVII - 2 http://i.imgur.com/X27lVJ5.jpg
FFVII - 3 http://i.imgur.com/fYwxIBY.png
Seriously this needs to be done. If it is possible to play pre-rendered PS1 games like this (and well, realtime would also benefit) it just must be done. In worst case I could pay a little if the hussle is too much. Here is a URL to a Google translated Waifu2x site
https://translate.googleusercontent.com/translate_c?depth=1&hl=en&prev=search&rurl=translate.google.se&sl=ja&u=https://github.com/lltcggie/waifu2x-caffe/releases&usg=ALkJrhivu32FwUwuPxodRAFRTS4bRFPimA
/Mike
@Hyllian made something called NEDI which tries to look like this.
Anyway, a real port of Waifu2x is impossible with the current shader spec. It is a neural network algorithm-driven filter. You'd need OpenCL, CUDA, compute GL/Vulkan to replicate something like that.
There is a GLSL implementation, but it's still too slow for real-time generation for games. It's more practical when used as a massive batch operation for an external texture pack (which Mednafen-PSX obviously doesn't support and probably won't support)
Also a lot of the PSX texture data are in chunks and sometimes really too posterized/aliased for waifu2x to be effective on (it only really shines with antialiased broad strokes, such as........ animes)
Texture replacement in Mednafen/Beetle PSX might be a fair proposition when we are further ahead with the GL renderer. Right now the main priority with the GL renderer is fixing the bugs and getting rid of the bottlenecks.
Hmm, looks like that repo only has a MSVC solution. I will try if I can get it to compile and run on Linux/OSX.
Thanx for your answers. I guess I'll have to wait longer for something like Waifu on PS1 games.
Waifu2x in even PC games with the proper backend support isn't practical either. I forgot to mention it's also bad with tiled textures.
Maybe it works better on pictures from games than it would in the game itself real time? I don't know... I just... need to have something that can make my old pre-rendered rpg's to look like the pictures I have shown. I have tried with ReShade but I can't get it to look like that. Maybe I would need a denoise filter with it I don't know..
I think Hyllian's shaders are going to be your best bet for this in real time. Check out the fast-bilateral-super-xbr presets: https://github.com/libretro/common-shaders/tree/master/denoisers (here's a pic) or the aforementioned NEDI (here's a pic). If you're using increased internal resolution, you could also check out the super-xbr-3d presets, which only apply the effect to the prerendered backgrounds/HUD and not the uprezzed polygons.
Thank you. But I don't know. From what I have seen so far it very far away from what I want. I could just try it but Retroarch gave me a headache last time so I deleted everything. I mean I don't know if it's worth it. Will it give me anything close to this quality with those shaders? http://i.imgur.com/fYwxIBY.png Because so far the most I have seen is kinda messy, sadly :(
I took a screenshot from FF7 and ran it through waifu2x, which runs slower than real-time, and then did the same thing with a shader preset I just made (nedi/fast-bilateral-nedi.cgp, also at 2x scale to match waifu; runs full speed on Intel HD3000 gfx) and here's the result, unlabeled so you can give them the pepsi challenge: http://imgur.com/a/ixTqZ/all
The right one is Waifu because it has less artifacts. But sure, just there the difference is not that big, at least not on that certain background. But try some other places and you will see. I mean take this as an example
http://i.imgur.com/X27lVJ5.jpg
It will absolutely not look like that, at least not with the filters I have tried. Maybe you can try it with the filter you made?
Even that glsl implementation isn't pure shader. The model handler is in C# or C++ and the model is passed to the shader by uniforms.
Not possible with current Retroarch shader specs.
On Sun, May 29, 2016 at 4:15 AM, Mickevincent [email protected] wrote:
The right one is Waifu because it has less artifacts. But sure, just there the difference is not that big. But try some other places and you will see. I mean take this as an example
http://i.imgur.com/X27lVJ5.jpg
It will absolutely not look like that, at least not with the filters I have tried. Maybe you can try it with the filter you made?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/libretro/common-shaders/issues/75#issuecomment-222346541, or mute the thread https://github.com/notifications/unsubscribe/AAyZbG0tIgLNg_4M9aa71dtsyMqPy5UIks5qGTz7gaJpZM4IpHsI .
How about if one made a complete new gpu plugin for ps1 emulation and then implemented some kind of waifu filter?
That's a pretty tall order, and it still wouldn't run anywhere near real-time.
Anyway, here are a couple more comparisons between waifu and fast-bilateral-nedi: http://imgur.com/a/GX3qU/all http://imgur.com/a/TZT9Q/all There are definitely differences--every upscaler has a signature, and waifu's is quite pleasant--but the differences are pretty small, particularly considering the speed difference. It's also important to recognize that the "artifacts" in the fast-bilateral-nedi shots are actually preserved details that waifu blasts out in favor of flat gradients and sweeping lines. Waifu is just not designed for maintaining detail in this sort of application, though its straight, smooth outlines and high contrast look nice.
That is actualy rather nice. Is it possible for you to upload the exact shaders for me? Is it retroarch? I dont know if I can get it to work but I'll try.
So I tried to upscale the same picture and it do looks better. I do it with these settings
- denoise and magnify
- Denoise level 3
- Model 2d-illust
Here it is with some others
http://imgur.com/a/zzdBm
I have never seen these backgrounds before in such high quality. Such an amazing filter.
It is indeed RetroArch, and you can download the shader preset through RetroArch's online updater (once you update the Cg shaders, it's in the 'nedi' subdir).
Those shots look nice but you're losing a ton of detail to posterization, and it looks like there's a lot of color post-processing going on. You can get reasonably close to that in RA by adding in the 'natural vision' shader and cranking up the bilateral blur: http://imgur.com/a/hJo6E/all But anyway, the long and short is: waifu2x can't run real-time. If you want to get pretty close, this is how you do it. If not, you're out of luck.
I have talked with the developer of waifu. He is gonna release a 2x faster version. Look its not that I don't believe you but I can't let go of the Waifu look. Someone said pherhaps, maybe, with alot of optimisation, changes beeing done and a strong gpu, it could (maybe) run. If you are up for it, even tho you bassicly said that it cant be done, I'am opening a Kickstarter with a reward for whoever does this. Of course it might not even be approved. Altough the Glide64 driver was approved. This is probably my last try. So just say no in worst case :)
To clarify, he did say it still wouldn't be good for real-time. Real-time neural network scaling is way, WAY easier said than done and it's not as trivial as you think it is. A Kickstarter won't help.
I want it because it looks better than anything else I've seen. But if it really is impossible there is not much to do. Thanx for your answers.
These are screenshots of FF7 taken with the GL renderer set to 1x and with the NEDI shader preset selected.
http://imgur.com/a/WCNM2
Looks reasonably close to Waifu2x and at least it's rendering in real-time.
Thanx, looks nice. I did this with some shaders in retroarch and reshade. The noise from the debanding gets very visable in pictures thou http://i.imgur.com/LGTynKw.png http://i.imgur.com/sIw1gPk.png http://i.imgur.com/XahR1ZA.jpg
Since Waifu2x is so slow, it is not possible in real time. However, this is supposedly possible to run in real time quite easily and could be, as I understand, as good as waifu2x. So is it possible that anyone could make this into a scaler/shader for retroarch?
http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html Codes and datasets are furthest down on the page.
Looking forward an answer =) /Mike
It may be possible but it's not going to look like waifu2x since it's a different algorithm trained on a different dataset. I'll keep an eye on it, though, in case we can do something with it in the future.
We did just get a nice port of NNEDI3, which looks very good, if you want to check it out.
Thanx for your answer hizzlekizzle. As I have understood it, it would be possible to train it to get the same look, or at least a similar. But yeah I have tried nnedi3. Sadly, then I prefer Super-Xbrz for pre-rendered games.