Very recently, there is a wave of interest among the Japanese community in developing a new algorithm for upscaling: waifu2x
The basic idea of this algorithm was developed in the Chinese University of Hong Kong, but only recently a Japanese implemented it in LUA and uploaded it to GitHub:
(... and it starts getting viral)
This original version use CUDA on Linux, and this seems to be problematic for many users...
Then shortly afterwards, people starts porting it to Windows, and with GUI. These have no CUDA acceleration however.
First Windows port (only upscaling, no training part):
GUI All-in-one package:
(Click on the floppy icon to download)
To use the GUI, launch waifu2x_win_koroshell.exe, then drag-and-drop your image file on to the main window.
Someone made an Avisynth port, but reportedly too slow to be of any use (0.02fps on Core i5):
I ported an OpenGL version to AviUtl:
GUI CUDA Version (some what troublesome):
(download link at the bottom, ~130MB!)
This GUI is Japanese-only. I have made another multilingual GUI that wraps it. For details see post 41:
Integration of AVX/FMA/CUDA/OCL...:
AviUtl Plugin that use the above DLL:
The release package already contains the DLL.
↑At version 1.4.1, this should be the fastest AviUtl version if using with CUDA.
@640x360 -> 1280x720 takes ~3 sec on GTX970OC(GigaByte)
Note that earlier versions may have problem with CUDA acceleration.
VapourSynth Plugin(also use w2xc dll)
Here are some samples:
1. Fringe pattern. Left: Original 256x256 px. Right: Waifu2x double sized, no denoise. Quite accurate and no artifact IMO.
2. Japanese-style animation image (a capture from the PC game, Da Capo). You can try the doubling the image using conventional bicubic, sinc or spline, you should notice the edges in the waifu2x images are a lot crisper.
+ Reply to Thread
Results 1 to 30 of 45
Last edited by MaverickTse; 15th Dec 2015 at 19:55. Reason: additionStopping development until someone save me from poverty or get me out of Hong Kong...
It's very slow, but does a good job.
I don't understand Japanese, but it's a simple matter of drag and dropping the image into the program window. It saves new image as a PNG in the same folder as original.Got my retirement plans all set. Looks like I only have to work another 5 years after I die........
Stopping development until someone save me from poverty or get me out of Hong Kong...
Last edited by MaverickTse; 15th Dec 2015 at 20:01.Stopping development until someone save me from poverty or get me out of Hong Kong...
To use the cuDNN acceleration, you need to sign-up the CUDA developer program on Nvidia's site and install the cuDNN package... quite troublesome.
It is fine to left out cuDNN IMO.
You can use the AviUtl plugin for the time being, if you don't mind a slow 0.1~0.2fps for upscaling from NTSC/PAL to double size.
Currently only the CUDA implementation may reach 1fps, but building that binary is as hard as building FFmpeg (and far less documentation).
There is a CLI version for the CUDA one, and of course in English.
I'm not sure this would be a good option for video (too slow), but it is a great option for images. It would make a good free alternative to Genuine Fractals, for example.Got my retirement plans all set. Looks like I only have to work another 5 years after I die........
All those examples seem to be of limited palette/anime. Any real-world/natural images to compare?
Here is a before / after on a jpg image I downloaded somewhere. The top one is the 2x upscaled version.Got my retirement plans all set. Looks like I only have to work another 5 years after I die........
Well, that's a very pleasant UPsize effect!
Slow as it is for video, there are a few projects where that could REALLY help/fix things. Good to know, thanks.
Neural networks for upscaling....
Okay, how is the network trained and how is it decided what looks go and what not for each training sample?
Since very few of us here can read Japanese, that might be a little difficult to answer. Perhaps MaverickTse has some info...
Last edited by newpball; 3rd Jun 2015 at 20:50.
Works here on PNGs. It gives some warning box that I can't understand, but click OK and it goes to work.
This program uses a seven step process, so it can take a long time depending on image size. The results are very good I think.Got my retirement plans all set. Looks like I only have to work another 5 years after I die........
And more importantly:
even though the GUI works well under Windows XP,
the command-line .EXE does not
My main PC got a mysterious short-circuit and the new rig will not be available until 13th June
(DDR4 RAM out-of-stock)
Limited support until then.
For how the algorithm actually works(in English), please visit the first GitHub link in the 1st post...
I cannot really answer that myself since I do not know what the heck is "convolution".
All i can tell is that everything works in YUV, and the heavy algorithm is only applied to Y. The UV planes are simple Bicubic resize.
Any magnification not 2X or 4X will involve extra shrinking step(area average or nearest neighbor)... so upscaling to 1.6X is actually slower than 2X
beware that on some images, waifu2x will create an excessively "oil-painting" feel, so that is why it is claimed to be designed for Anime-style images.
You can train the algorithm using your own image set, but you need a Nvidia rig on Linux, since only the Linux version has the trainig component.
Hopefully developers around the world will join the quest in making it faster...
Neural networks are finally feasible? About goddamn time.
This looks very promising, for animation at least.
I upscaled the first episode of Death Note to 960p.. I cant provide any links here, but you can surely find it somewhere if you search for it.
Upload a 30 second clip of it. I'd like to see it. How did you even convert a whole episode? It took me half a minute just to upscale ONE 480p image. A whole episode would take what, 10 days?
I also try this one and i must say it works sometimes nice, sometimes not and yes it is slow but i don't expect speed from non optimized code - works more than good as a proof of concept.
The quality of your death note episode is not too good. It doesn't look any better than a substandard upscaler designed for cartoons.
I'm guessing you used a horribly inefficient version of waifu2x that allowed it to be that fast. Send me one original frame of your video and I'll see if the waifi i'm using will do any better.
And here is some upscaled frames: http://imgur.com/a/KHCTa
I used the one I said, it uses CUDA acceleration, thats why its fast. I also upscaled it 2x with 2x denoise. Maybe thats why it doesnt look as good as you want.
Also used the already trained waifu2x from the creator. Since I dont have enough ram to train it my self.
I heard that now they trained it better, it may yield better results now.
Also, if we were to train it extensively.. It would result in much better results..
Those photos are 580p, show me the 960p ones.