Uncompressed MKV?

[TheAlmighty]

I have created my AVS file, in which I use AvsPmod to do all my editing.


LoadPlugin("F:\-----\Downloads\Encoding Software\MeGUI_2356_x86\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("C:\Users\-----\Videos\SAMURAI SHAMPLOO V1 E1.d2v", cpu=6, info=3)
LoadPlugin("F:\-----\Downloads\Encoding Software\MeGUI_2356_x86\tools\avisynth_plugin\Colo rMatrix.dll")
ColorMatrix(hints=true, interlaced=true, threads=0)
Load_Stdcall_Plugin("F:\-----\Downloads\Encoding Software\MeGUI_2356_x86\tools\yadif\yadif.dll")
Yadif(order=1)
crop(2, 6, -2, -4)
Lanczos4Resize(1024,576)
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\TTempSmooth.dll")
TTempSmooth()
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LSFmod.v1.9.avsi")
LSFmod(strength=120)
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\gradfun2db.dll")
gradfun2dbmod(thr=3,str=0,mask=false)



So how the above AVS file looks in AvsPmod, is exactly how I want the video to look. I use MeGUI to encode to MKV, but when I encode the video, it seems as though it was pointless even going through the effort of adding filters. I've done so many tests and they all look the same - Watchable, and most people wouldn't complain, but I edited the video for a reason and it's not the result I wanted. My video is anime and the encoder setting is unrestricted 2pass Anime_Toon HQ, but I have tried others. I've played around with a few of settings within the profiles, but I don't seem to have much luck. I understand compression is the reason for this, so if someone can perhaps enlighten me, that would help me heaps. All I am trying to do is keep what I see in AvsPmod and encode with the least amount of quality loss as possible.

[sanlyn]

Without a sample of your original unprocessed input, and without more info about what you want for output, there's not much anyone can advise. You can post samples of several seconds of unprocessed source video (choose a scene that involves motion) here or at file sharing sites. You appear to be working with MPEG source. You can use the free DGIndex utility to cut a sample m2v video -- audio isn't required.

GradFun2DBmod: thr=3 is pretty strong stuff. Apparently you have reasons for using that setting, but one can only guess what they could be. No one knows what you're working with.

[TheAlmighty]

AVI Uncompressed.png - AVI uncompressed
MKV.png - Finished MKV

Both encoded using the same AVS file.

[poisondeathray]

why are your screenshots 1920x1080 ?

yadif is probably not an ideal filter to use here - you would probably IVTC most anime, not deinterlace . It's responsible for those ugly line artifacts that you've upscaled in the screenshot

what encoding settings did you use? use better encoding settings and lower crf settings, and it will eventually approach the uncompressed version

[sanlyn]

We have to advise based on your source video, not on reprocessd frame caps. You might have upsized to demo the banding, but that only demonstrates how resizing affected the original problem. Otherwise you'll get very general answers, some of which might not even apply. I'll second the notes in poisondeathray's post.

One general caution: denoise first, resize last. Why resize noise? Life is too short for cleaning that up.

Color Banding and Noise Removal

[TheAlmighty]

Thanks very much for the quick replies. Look, it's not a huge issue. I'm still an amateur. Trial and error. Perhaps I should just accept what it is and carry on.

[sanlyn]

As you wish. What you describe so far are common problems that amateurs regularly repair with widely accepted methods. Yes: it's a learning process. You can't expect to keep doing the same thing and get a different result.

[poisondeathray]

In megui, those presets are just "starting points" . You have to adjust them (especially the crf value; lower yields higher quality but larger filesizes)

If you want to learn how to improve what you are doing - people are willing to help but you have to post an unprocessed sample of the source - I'm betting megui suggested the wrong script by choosing yadif . Anime is almost never interlaced . And even if there was a reason to deinterlace (maybe it's field blended source, and you need to deblend), there are better, higher quality deinterlacers to choose from- you can see the line artifacts generated by yadif upscaled in your screenshots

[TheAlmighty]

Unprocessed Video.mpg

[sanlyn]

Thanks. Rather late at night here, but we appreciate your effort. The mpg appears to have been through field-blend deinterlace routines -- which is a shame, because it used to be progressive with applied pulldown (telecine). I can tackle more of this in the morning, but I'm sure members might chime in.

[sanlyn]

Unprocessed Video.mpg

the best I've been able to obtain so far was with this:

MPEG2Source("path\to\d2v project\Unprocessed Video.d2v")
AssumeTFF().QTGMC(preset="very fast")
sRestore(frate=23.976)

But there are still some remnants of telecine or blends, so I haven't yet figured out the blending pattern. Will keep working on that if no one makes better suggestions. Meanwhile, I'm looking at some better noise reduction that will help prevent the macroblocks and banding you're having trouble with.

[jagabo]

He's not asking about how to process his video:

So how the above AVS file looks in AvsPmod, is exactly how I want the video to look.

He's asking how to encode it to preserve what he's got. I didn't look that closely at the two sample images but the encoded video has noticeable banding in the dark areas. That is one of the weaknesses of x264. The two approaches to preventing that are to retain noise (use a CRF around 12, or very high bitrates, or aq-strength=1.8 or higher) or use 10 bit x264. But hardly anything outside a computer can play 10 bit x264.

[TheAlmighty]

Comparison1.jpg

This is what I get when I change the QF setting to 12.

Comparison2.jpg

This is what I get when I leave the QF at the default settings.

In Comparison1, there is still some noticable banding/artifacts. Will that improve if I play around with my script or is that the best I'm going to get?

[jagabo]

You can try adding a little more grain/dither to your script. To make up for what's lost due to compression.

[manono]

So how the above AVS file looks in AvsPmod, is exactly how I want the video to look.

No you don't. You just think you do.

He's not asking about how to process his video...

He also said he's a beginner. Unless he buys the excellent NTSC versions, it has to be unblended as best as possible or none of the other advice will help much. As pdr mentioned earlier, you don't just deinterlace something like this. In my opinion.

Sanlyn, it's 23.976. Or, just leave out 'frate=23.976' and the result will be 23.976 automatically. Yes, the sample doesn't unblend all that well, although I suspect the body of the episodes will be better.

[sanlyn]

Apparently the titles use a different blend pattern. I suspect they were originally true interlace (?), the main body telecined. Anyway, sRestore made it useable.

@jagabo, I think the O.P. wants to know how to "process" this video. The code he posted is resizing to 1024x576 (before it's denoised). For DVD or BD/AVCHD compliance, that frame size won't work. It'll be resized again on encoding unless the O.P. really wants 1024x576 for output. The source is already full-frame 720x576 at a 16:9 display ratio and has very slight letterbox bars. Why resize ? ? ? ? I don't think the O.P. realizes that this movie wasn't produced as 16:9 but as 1.8888:1, so he's squishing his video and making it more narrow-screen than the original. Maybe he likes working this hard to distort his movies, even if by a little, but I think we might advise that the only movies made at 16:9 are made for TV and by consumer video cams.

I found that using the dither plugin functions such as GradFun3 and its other features is a big help with banding problems, especially with anime. The O.P. is trying to remove all the grain with smoothers and then play with CRF. Result = banding.

I don't know why we're trying to work with screen caps from VLC Player. VLC displays banding in anime even when no banding is there.

[TheAlmighty]

I feel as if I'm being indirectly mocked. I've already clarified I'm no expert. The only reason I'm doing the things I am is because I've read up on it. Perhaps I'm doing all the wrong things, but the only reason I've gotten this far is through trial and error. Ultimately I would like a nice clean anime, and really knowledge is power. So far I appreciate the help... Honestly. If you want a better sample I can provide you with one - Maybe of the section similar to the screenshots? I'll even post the dgindex log.

[manono]

Maybe of the section similar to the screenshots? I'll even post the dgindex log.

Useless.

If you want a better sample I can provide you with one

One with steady movement taken from the main part of one of the episodes might be more representative, yes.

[hello_hello]

My video is anime and the encoder setting is unrestricted 2pass Anime_Toon HQ, but I have tried others. I've played around with a few of settings within the profiles, but I don't seem to have much luck. I understand compression is the reason for this, so if someone can perhaps enlighten me, that would help me heaps. All I am trying to do is keep what I see in AvsPmod and encode with the least amount of quality loss as possible.

I haven't read the whole thread, but I thought I'd point out you appear to be using some very old and obsolete x264 encoder presets (unless they're Xvid encoder presets).
Try opening the x264 encoder configuration and loading the defaults. Select "Animation" in the Tuning dropdown box and lower the Quality setting if need be (lower values = higher quality). The default is 23, but I think people around here commonly use values between 18 and 22. I generally use 18 myself. If you add the video encoding job to the queue directly, that'll run a single pass, quality based encode. The file size will be unknown (it depends how hard the video is to compress).

If you're using 2 pass encoding (ie specifying a file size) then the quality becomes the unknown factor. That's nothing specifically to do with the preset you're using... those x264 encoder presets haven't been in use for quite some time.

[sanlyn]

I feel as if I'm being indirectly mocked. I've already clarified I'm no expert. The only reason I'm doing the things I am is because I've read up on it.

Oh, my, mocking? Nothing of the sort, and certainly not intented. Besides, precious few members were experts on their first visit here. I hope to point out that, yes, you are doing a few things incorrectly -- or, let's say, there are better ways of doing it.

Jagabo and others appear to be correct about the way some x264 settings can help get cleaner results. My suggestion is that more careful prep before the encoding step would help as well. I'd say that in this case, cropping + resizing to 16x9 is both unnecessary and, strictly speaking if you consider the original image aspect ratio, it would alter the aspect ratio of the original image if only slightly. If you expect to make standard DVD or even BluRay or AVCHD for disc from the result, then 1024x576 would be resized anyway back to 720x576 to comply with standard encoding and players. And even careful resizing procedures can incur quality issues.

Your suggestion that you submit an original sample of the "problem scene" you've been discussing would be helpful. Your original sample has many good examples of some of the video's structural problems. But if we're all working on the same specific issue, things could move along with more clarity.

[sanlyn]

Ooops, I said the movie is 1.888:1. Sorry, the 1999 movie was created at 1.85:1. That's wider than 16x9 ( which is 1:777778:1).

[TheAlmighty]

sanlyn - Sorry for being arrogant.

Here is the sample which is more relevant to the pictures.


Unprocessed Video2.m2v


I have provided a photo of the Info Log. Probably not important, but provided nonetheless.


Info Log.png

[sanlyn]

Thanks for sample 2. Will be spending some time with it. Meanwhile, you didn't seem arrogant -- not from my viewpoint, anyway. Hope you were not offended -- perhaps it results from a habit we Yanks seem to have, which some people refer to as murdering the language. And often many of us do tend to, mmm, get to the point around here.

From this sample, I see what you mean. Still captures didn't accurately relay your problem -- things look quite different in motion. Offhand I think some of the excessive grain was purposeful, to avoid banding. However, it seems that the style of grain added was more coarse than needed. Difficult to say for certain. This sort of thing appears often with anime, even with many highly regarded issues. Sometime it can be fixed, sometimes not; sometimes it comes down to compromise. It's definitely more visible here than in your first sample. I'll try a few recommended tricks and post the results. I'm certain other members will pounce on this one.

[jagabo]

Too much field blending. I'd toss it and find another source. It's even available on Blu-ray (though apparently upscaled from SD). Why waste your time?

[sanlyn]

I just found some new issues, but jagabo beat me to it.

Yes, the field blending is a turnoff. Much of it seems irreparable. But probably a good lesson in dealing with grain+banding.

[sanlyn]

Unprocessed Video2.m2v

Nice sample. Gave us plenty to think about.

Well then, no one has devised the perfect way to 100% unblend this puppy, so I proceeded to work with what sRestore could do and focussed on the banding and resizing issues. The attached mkv is 23.976 fps (add pulldown flags if you want 25FPS). It's been resized and plays at the movie's original aspect ratio. It's 1:85:1, not 16x9. The sample appears to have been heavily processed earlier, but I saved as much detail as I thought possible while cleaning up banding and block noise. Hope the sample script offers a few ideas.

# #=== Avisynth pluygins used:
#- QTGMC-3.32.avsi
#- srestore.avs
#- SnoothLevels (in SmoothAdjust.dll)
#- MCTemporalDenoise.avsi
#- aWarpSharp.dll
#- FastLineDarken 1.3.avsi
#- LSFmod.avsi
#- Dither.avsi
#- dither.dll
#- mt_xxpand_multi.avsi
#- avstp.dll
#- mt_masktools-26.dll
#- masktools.dll
#- GradFun3 (in the dither plugins package)

MPEG2Source("Path\to\project\Unprocessed2.d2v")
AssumeTFF().QTGMC(preset="very fast")
sRestore()
SmoothTweak(saturation=1.1)
MCTemporalDenoise(settings="low") 
MergeChroma(aWarpSharp2(depth=30))
FastLineDarken()
LSFmod(defaults="slow")
GradFun3(thr=0.7,mask=1)
Crop(0,4,0,-4)
Spline36Resize(1280,688).AddBorders(0,16,0,16)
return last