Help - Best way to encode anime and improve image quality

[yusuke10]

Hi all, I've been reading about the ways to encode using Intel Quick Sync/GPU/x264.

So far seems x264 got the best quality if I'm not wrong. Is it true? I want to encode in x264 too for compatibility.

I've a anime in 720p that I want to encode in the best quality possible, so far I'm using x265.

What's the best software to convert x265? I'm using MeGUI 32 bit with AviSynth 64 MT with x265 1.8.
When seeking a part of the video, I get strange artifacts only in x265 on the result video, like these:



Is it normal?


And there are some things that I want to improve in the video:

1-Jagged edges = I'm using SharpAAMCmod 4 times, it improves a bit but I've seen results better than mine.

2-Image seems a little blur = want to get it more crisp to get more details.

3-Colors seems a little bit off = used AutoAdjust to get it more brighter and more contrast, but still the colors seems too colorful now.

These are the settings I'm using:

AnimeIVTC(mode=1,ifade=true,killcomb=3,aa=0,aatype="EEDI2",dark=1,thin=0,sharp=150,smooth=100,stabilize=true,tradius=3,aapel=4,aablk=16,aaov=8,overlap=8,pel=4,bbob=4,degrain=3)
AutoAdjust(threads_count=8,avg_safety=0.0,balance_str=1.0,auto_gain=true,bright_limit=1.25,chroma_process=50)
SharpAAMCmod(dark=1,smooth=100,thin=0).SharpAAMCmod(dark=1,smooth=100,thin=0).SharpAAMCmod(dark=0,smooth=100,thin=0).SharpAAMCmod(dark=0,smooth=100,thin=0)

These are the original video and the samples, all of them 30 seconds each:

Original

x264 with above settings

x265 with above settings

With QTGMC(InputType=1, Preset="slow", EZDenoise=2.0) intead of AnimeIVTC



I did see some anime remaster and upscale from a guy in my country and it's excellent(only a few people do it here), but seems like he's inactive right now.
Example of his work:
http://screenshotcomparison.com/comparison/164653/picture:0
http://screenshotcomparison.com/comparison/164653/picture:1

I really appreciate any help given =)
I have spent days trying to figure out how did other people get these amazing results.

Sorry for my english, I'm brazilian and I didn't have any English lesson but really want to finish subbing my favorite anime with the best quality possible.

Thank you for your help guys!

[Selur]

the mega-links require a crypto key for access,....

[yusuke10]

the mega-links require a crypto key for access,....

Sorry Selur, now it's updated.

Thanks.

[sneaker]

I think it's a problem of MeGUI's mp4box version - MeGUI devs should update it. You can download a nightly version from here. Extract using e.g. 7zip and replace files in MeGUI\tools\mp4box folder.

(x265 is also quite old, btw)

/edit:
Both mp4box and x265 have now been updated on the MeGUI development update server.

[yusuke10]

I think it's a problem of MeGUI's mp4box version - MeGUI devs should update it. You can download a nightly version from here. Extract using e.g. 7zip and replace files in MeGUI\tools\mp4box folder.

(x265 is also quite old, btw)

/edit:
Both mp4box and x265 have now been updated on the MeGUI development update server.

Thank you sneaker, I was using the stable server, changed to development and updated everything.
Now the x265 doesn't have those artifacts like before, and video seeking for x264 and x265 seems to be accurate.(Before it moved automatically to keyframes)
Updated the links for the encoded versions of both if any use to you guys.

Thanks for the replies.

Can someone give me an advice on how to improve visual quality? Been trying it really hard, but I'm stuck.

[Selur]

for the fun of it:

SetMemoryMax(1536)
SetMTMode(5,8) # change MT mode
LoadPlugin("G:\Hybrid\avisynthPlugins\LoadDll.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\dfttest.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\SangNom2.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\fturn-26.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\mt_masktools-26.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\flash3kyuu_deband.dll")
LoadDll("G:\Hybrid\avisynthPlugins\libfftw3f-3.dll")
Import("G:\Hybrid\avisynthPlugins\EdgeSmoothFast.avs")
Import("G:\Hybrid\avisynthPlugins\Hysteria.avsi")
LoadCPlugin("G:\Hybrid\avisynthPlugins\ffms2.dll")
# loading source: C:\Users\Selur\Desktop\Yu-Gi-Oh! 5D's 001 - original.mkv
FFVideoSource("C:\Users\Selur\Desktop\YU-GI-~1.MKV",cachefile="H:\Temp\mkv_583188236f6f903fa4e2ea21993bcfe4_4827_1_0.ffindex")
# denoising
SetMTMode(2) # change MT mode
dfttest()
# anti-aliasing
EdgeSmoothFast(strength=60,hiQ=true,twoway=true)
# line darkening
Hysteria(strength=1.3)
# debanding
f3kdb(keep_tv_range=true)
distributor()
return last

Also there is some UGLY ghosting in the scene from ~620 to ~720 which should be dealt with separately (using trim and some deghosting)
Ghosting is visible in some other scenes as well but not so obvious.

[yusuke10]

for the fun of it:

SetMemoryMax(1536)
SetMTMode(5,8) # change MT mode
LoadPlugin("G:\Hybrid\avisynthPlugins\LoadDll.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\dfttest.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\SangNom2.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\fturn-26.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\mt_masktools-26.dll")
LoadPlugin("G:\Hybrid\avisynthPlugins\flash3kyuu_deband.dll")
LoadDll("G:\Hybrid\avisynthPlugins\libfftw3f-3.dll")
Import("G:\Hybrid\avisynthPlugins\EdgeSmoothFast.avs")
Import("G:\Hybrid\avisynthPlugins\Hysteria.avsi")
LoadCPlugin("G:\Hybrid\avisynthPlugins\ffms2.dll")
# loading source: C:\Users\Selur\Desktop\Yu-Gi-Oh! 5D's 001 - original.mkv
FFVideoSource("C:\Users\Selur\Desktop\YU-GI-~1.MKV",cachefile="H:\Temp\mkv_583188236f6f903fa4e2ea21993bcfe4_4827_1_0.ffindex")
# denoising
SetMTMode(2) # change MT mode
dfttest()
# anti-aliasing
EdgeSmoothFast(strength=60,hiQ=true,twoway=true)
# line darkening
Hysteria(strength=1.3)
# debanding
f3kdb(keep_tv_range=true)
distributor()
return last

Also there is some UGLY ghosting in the scene from ~620 to ~720 which should be dealt with separately (using trim and some deghosting)
Ghosting is visible in some other scenes as well but not so obvious.

Selur, thanks for this Script, tried it and it's very good.

Is there a way to make the image look more crisp?

And what filter do you recommend for deghosting?

[Selur]

Is there a way to make the image look more crisp?

a. tweak the settings of Hysteria
b. use a sharpener after the anti-aliasing

And what filter do you recommend for deghosting

Sorry, I rarely use filters for de-ghosting so I can really recommend anything.
First I would probably try unblend() and EraseGhost. (but remember to only use these on the scenes that are affected)