Noise in picture

[brusno]

What is this kind of noise called? And can it be removed by any avisynth filter?
Or is it just artifacts because of the quality?
I'm referring to what is in the drawing borders, ie. round the scarlett ribbon, mouth and so on.

[davexnet]

I believe it's mostly block noise due to low bit rate.
Perhaps a de-blocking filter?

[poisondeathray]

Yes, these are due to low bitrate encoding. DCT ringing and block noise, some chroma noise

Yes you can improve it with various avisynth filters such as line thinners, antialiasing, denoisers -- BUT -- you should to post an original sample , not a jpeg image (jpeg is lossy, you don't know which are jpeg artifacts vs. original artifacts), and not a resized image (resizing can change the spatial relationships of the artifacts)

[sanlyn]

Difficult to say, really. The image is lossy JPG, which adds its own artifacts to images. Which of the problems (macroblocks, mottling, mosquito noise around fading and uneven-density lines) do you think belong to your original video, and which belong to jpg compression? Block noise is in the flat red areas. The image size has us guessing (1280x1024 is .8:1 image ratio, but isn't a stanadrd BD or DVD image size, so one would guess that resizing might have occurred. But that's just guessing. The video's original source is unknown.

One thing the O.P. apparently doesn't realize is temporal noise changes from frame to frame. But I'd venture to say that if any three frames from the video were posted, the noise would look different in each image.

[jagabo]

If the source is MPEG start with DgIndex and Mpeg2Source("filename.d2v", CPU=6). That will deblock and dering.

[brusno]

Ok, I'll see if I can cut out a sample of it, with motion pictures.
How do I insert a video clip? or must it be via youtube, facebook or dailymotion?
The source is mpg2 (DVD, PAL, 4:3), and the conversion is mp4, 4:3, 1200 kb/s, 2-pass.
The size of the picture is because it is "print screen", and then it get what my screen resolution is.

[jagabo]

Use DgIndex to export a clip as an elementary stream. Mark-in, mark-out, File -> Save Project And Demux Video. Or Mpg2Cut2 to export as an MPG file. Mark-in, mark-out, File -> Save This Clip.

Upload the clip here, not to a service like Youtube which will reencode the video.

[brusno]

sample of mp4 file:
Ichabod And Mr-1_sample.mp4

[jagabo]

A reencoded sample isn't helpful. Demux from your original source file.

But using your mp4 sample, try Deblock_QED().HqDeRing().UnDot(). Also adding McTemporalDenoise(settings="high") and FastLineDarkenMod() will help with animated material like that. And RemoveSpots() to remove scratch-type spots.

[brusno]

A mpg2 clip from the source (with DGIndex):
Toad_mpg2.demuxed.m2v


I also have to mention that the mp4 is made in Handbrake with "Detelecine" and "Decomb" on default.

[sanlyn]

Your source MPEG is not interlaced, not telecined, and has no combing. It might play as "interlaced" or perhaps was encoded that way, but playing field by field you can see that it is progressive 25 FPS. Perhaps our video production experts can give some insight into that.

It's a bit noisy and has some flicker. Any more info on where the MPEG came from? Is it DVD, re-recorded DVD, or what?

[brusno]

The source is PAL DVD, 4:3, original and it has been ripped with DVDShrink. Maybe DVDShrink do some de-interlacing when ripping it? It is ripped "movie only".

[jagabo]

Your source MPEG is not interlaced, not telecined, and has no combing. It might play as "interlaced" or perhaps was encoded that way, but playing field by field you can see that it is progressive 25 FPS. Perhaps our video production experts can give some insight into that.

It's very common for PAL video to be encoded interlaced even though the frames are progressive. Especially when the source is video tape.

Here's the mpeg sample opened with Mpeg2Source() with CPU=0 and CPU=6 (no deblocking or deringing, full deblocking and deringing):



Adding UnDot() and MCTemporalDenoise(settings="high") gets rid of most of the rest:



Unfortunately, MCTD at high settings like that also removes some detail and smears some motions in low contrast areas (note the red carpet). That's frame 69 in the sample video.

[brusno]

When looking at the source (DVDShrink ripped) in the "Preview" (stillpictures) window in Staxrip it is been shown as interlaced. You can see the stripes very clearly. But in the "preview" in Handbrake (film) it doesn't show at all.

Stillpicture from Staxrip-preview:

[sanlyn]

When we say "source" video, we don't mean video that has been deinterlaced, deblocked, denoised, etc., etc. Source means "source".

As it is, your processed MPEG didn't need a great deal of work. But it wasn't the original source. Jagabo's cleaner image sample from the progressive MPEG2 looks fine to me.

[brusno]

@jagabo:

It was quite an improvement to the film that you show At the cost of loosing some other detail (carpet). But it's all about compromise isn't it? I'll try and do the things that you suggest to see if I can manage it.

@sanlyn:

All is relative. The source is the DVD, but before the source is available it has to go through a ripper first. And you never now what the ripper does to the film.

[jagabo]

When looking at the source (DVDShrink ripped) in the "Preview" (stillpictures) window in Staxrip it is been shown as interlaced. You can see the stripes very clearly. But in the "preview" in Handbrake (film) it doesn't show at all.

Stillpicture from Staxrip-preview:

[Attachment 16463 - Click to enlarge]

I don't know what StaxRip is showing but here are frames 828 and 829 from the M2V file:





I believe those are the two frames that match the fields in the Staxrip image. Though the video is encoded in interlaced mode all the frames in the M2V are progressive. Ie, each frame of the M2V file comes from a single film frame.

[sanlyn]

Rippers can be used without deinterlace or other options. As it is, your original image from post #1 shows that you resized a lossy compressed video, which always generates problems. Anyway, because you have some noise and other problems you should have converted your ripped MPEG to lossless medium for the work you planned to do, then re-encode afterwards. You cannot process properly using lossy compresseion.

[brusno]

Rippers can be used without deinterlace or other options. As it is, your original image from post #1 shows that you resized a lossy compressed video, which always generates problems. Anyway, because you have some noise and other problems you should have converted your ripped MPEG to lossless medium for the work you planned to do, then re-encode afterwards. You cannot process properly using lossy compresseion.

I have looked at ripps from both anyDVD and DVDShrink, and they look very equal to the source DVD. So I don't think the video has been altered, just prosessed. I don't think the rippers have deinterlaced it.
I have to correct you in your belief that I have resized the video. I am using anamorphic coding in Handbrake to keep the original size of the video. The stillpicture from post #1 does have another resolution than the video because I took a "print-screen" of the video when playing it in VLC. And this caused the picture to get 1280 x 1024, which is the resolution that I have on my pc. The video however has the resolution 720x576 (4:3, anamorphic, SAR=1.25, DAR=4:3). Even after ripping.

[brusno]

@jagabo:

Now I've managed to try Mpeg2Source( "", cpu=6) and undot(). I made a .d2v file with DGIndex first, and then I made a .avs file which pointed to the .d2v file:
MPEG2Source("D:/Toadman.d2v", CPU=6)
LoadPlugin("D:\UnDot.dll")
UnDot()

And then I opened the .avs file in the VirtualDubMod to see the improvements. And I could see improvements with only this 2 settings!
I will also try out MCTemporalDenoise(settings="high"), but I have to find the corresponding .dll file first.

But....how am I storing the mpg2 file with the improvements and the sound back in?
Guess I have to read some avisynth teori on this one. But do you have any tips for me on this?

[jagabo]

Now I've managed to try Mpeg2Source( "", cpu=6) and undot(). I made a .d2v file with DGIndex first, and then I made a .avs file which pointed to the .d2v file:
MPEG2Source("D:/Toadman.d2v", CPU=6)
LoadPlugin("D:\UnDot.dll")
UnDot()

And then I opened the .avs file in the VirtualDubMod to see the improvements. And I could see improvements with only this 2 settings!

Yes, I think Mpeg2Source(...,CPU=6) gets rid of maybe 3/4 of the DCT ringing dots. UnDot() gets maybe another 10 percent.

I will also try out MCTemporalDenoise(settings="high"), but I have to find the corresponding .dll file first.

See the all-in-one installer here: http://avisynth.org/mediawiki/MCTemporalDenoise
MCTD has a lot of parameters you can play around with...

But....how am I storing the mpg2 file with the improvements and the sound back in?

I usually use the x264 CLI on the AVS script to make a file with just the video.But you can use any encoder or conversion program that accepts AVS scripts (HCGUI for MPEG 2 for example). DgIndex should have demuxed the audio to an AC3 file. I usually just mux that audio with the new video. Since I'm mostly making MKV files these days I have x264 output an MKV then use MkvMergeGUI to mux the AC3 audio with that video.

[sanlyn]

I have to correct you in your belief that I have resized the video. I am using anamorphic coding in Handbrake to keep the original size of the video. The stillpicture from post #1 does have another resolution than the video because I took a "print-screen" of the video when playing it in VLC.

Sorry, I stand corrected. I didn't realize that you don't know how to make unmodified frame captures directly from video. If you need to know how, just ask.

I have looked at ripps from both anyDVD and DVDShrink, and they look very equal to the source DVD. So I don't think the video has been altered, just prosessed. I don't think the rippers have deinterlaced it.

Mm, look again. You can't go from the interlaced frame in post #14 to the frames shown by jagabo in post #17 without deinterlacing. As it is, your non-deinterlacer might have made some mistakes. Such as broken lines all over the place, one of them shown here (m2v, frame 744):


But those breaks might be in the original. We don't have the original, so can't say. But there were a few other booboo's (m2v, frame 888):


Your non-deinterlacer also propogated noise across multiple frames, including dots, So some of that junk can't be cleaned. Shimmer on camera movement is also a problem, due to "non-interlacing" without first doing a little denoising. Flicker is obvious in the m2v, which I'm certain did not exist on your original DVD.

mkv #1 posted below clears most of the chroma noise (except dots) and all of the flicker.
mkv #2 has more filtering to help smooth camera pans some more, but it smears detail.

[poisondeathray]

DVD shrink cannot deinterlace (doesn't have that ability)

If you used DVD Shrink with zero compression that is a 1:1 rip, identical to the source DVD. But if you used any % compression in the settings then it will transcode (not re-encode, there is a slight difference)

sanlyn - your first screenshot frame #744 - I don't see that "missing" line in the video ?. Weird. Am I looking at the same video ? I do see the blend on frame 888 . Maybe we are using different dgdecode.dll versions ?

[sanlyn]

Hmm, dgdecode. I'll check on that.

[brusno]

@poisondeathray:
Yes it's a 1:1 rip without compression, and I'm thinking too that it probably is identical to source. But it is pointed out that it contains some errors. Quality issues. One possibility is that even the source-DVD contains these errors.
But I'll upload an anyDVD rip in case DVD Shrink has done some modification to it. Then you can see if there is any difference between the 2 ripps. Because it is important to have the best possible base-material when coding it later.

AnyDVD-rip:
Toadripp_anyDVD.demuxed.m2v

@sanlyn:
The two clips that you uploaded look also very improved. Have you used avisynth script, filters too?
May I ask which?
And yes, I'd love to know how to make unmodified frame captures directly from video.

About the interlaced picture that I showed in post #14; seems that it is StaxRip which shows the film interlaced like this. So that I have to use deinterlacing to correct it. It is strange that only StaxRip show the film interlaced in preview, because it is using DGIndex too, when it is processing the source file (mpg2). Handbrake does not show the film visibly interlaced. So in that encoder, I don't have to use de-interlacing. This is a mystery.

And by the way, I'm using DGIndex version 1.5.8.

[sanlyn]

Well, I'm a bit mystified. First time I've seen something like the broken lines but they went away after a reboot. I should reboot more often anyway, as some filters have memory leaks that get RAM a bit constipated after a while.

There are a dozen ways to make direct frame captures. In the DGIndex preview window, use "Video..." then click "Copy frame to clipboard" (or hit F8). In VirtualDub, use the "Video" menu to copy the input frame or output frame. These methods copy to the Windows clipboard. Open a photo app, start a "new" photo and use "Paste" from whatever menu your photo app uses. Some photo apps have a "paste as new image" menu option, or something similar. If you want the cleanest image with no jpg compression artifacts, save the image as PNG (although an image this pristine is often not necessary). You can do the same thing in Windows' free mspaint. Just open mspaint, paste, and save. Paint can save PNG's.

I'm not intimately familiar with the apps you use. I've used some in the distant past. I guess the initial thrill of ripping encrypted stuff kinda wore off a long time ago.

For mkv #1:

MPEG2Source("[path]\Toad.d2v",CPU=5)
RemoveSpots()
DeDot()
FFt3Dfilter()
DeBlock_QED()
mergechroma(aWarpSharp2(depth=20))

#---- Deflick anti-flicker from VirtualDub -----
ConvertToRGB32(matrix="Rec601",interlaced=false)
vdpath="D:\VirtualDub\plugins\"
LoadVirtualDubPlugin(vdpath+"Deflick.vdf", "DeFlick",1)
DeFlick(0, 8, 0, 0, 0)
ConvertToYV12(interlaced=false)

For mkv #2 I used some slower and stronger stuff:

MPEG2Source([path]\"Toad.d2v",CPU=5)
RemoveSpots()
DeDot()
DeBlock_QED()
QTGMC(InputType=1,preset="fast",TR2=2,sharpness=0.7)
mergechroma(aWarpSharp2(depth=20))

ConvertToRGB32(matrix="Rec601",interlaced=false)
vdpath="D:\VirtualDub\plugins\"
LoadVirtualDubPlugin(vdpath+"Deflick.vdf", "DeFlick",1)
DeFlick(0, 8, 0, 0, 0)

Note that this output is saved as RGB32. Followed in VirtualDub with low-octane NeatVideo used mostly as a low-frequency smoother to combat motion shimmer on camera pans. But there are probably better way to handle that particular issue. After NeatVideo I reconverted to YV12 and encoded with TX264.

There are umpteen ways to do it. I declined to use MCTemporalDenoise because it required high settings that softened too much, and some of the other plugins are smoothers as well, so....adding MCTD was too much smoothing for my taste, at least in this case. Sometimes you just have to experiment with short scenes. Once you find the right combo, you're set.

[sanlyn]

sanlyn - your first screenshot frame #744 - I don't see that "missing" line in the video ?. Weird. Am I looking at the same video ? I do see the blend on frame 888 . Maybe we are using different dgdecode.dll versions ?

Found the culprit: RemoveSpots(). First time I've ever seen that happen.

[poisondeathray]

sanlyn - your first screenshot frame #744 - I don't see that "missing" line in the video ?. Weird. Am I looking at the same video ? I do see the blend on frame 888 . Maybe we are using different dgdecode.dll versions ?

Found the culprit: RemoveSpots(). First time I've ever seen that happen.

Wow, you must be joking or sarcastic ??

All the dirt/spot removal filters will do that (remove wanted objects) even with the strength turned way down

They are best used with compositing layers, and masks, never alone

[jagabo]

sanlyn - your first screenshot frame #744 - I don't see that "missing" line in the video ?. Weird. Am I looking at the same video ? I do see the blend on frame 888 . Maybe we are using different dgdecode.dll versions ?

Same here. Frame 744 looks ok (it isn't missing the line segment, and I didn't notice anything like it anywhere else). Frame 888 is a double exposure.

Ah, nevermind. I see you sorted it out. Yes, I see the same thing with RemoveSpots().

[jagabo]

In the original posted Toad_mpg2.demuxed.m2v, opened directly in VDub with no help from DGIndex, that image is frame 742. After DGIndex/DGDecode, it's frame 744 with a break in the rope.

The clip starts with two B frames. I guess the MPEG 2 decoder in VirtualDub ignores them whereas DgIndex substitutes the third frame (an I frame).