artifacting?

[Coffindan]

I'm not sure what to call it but I've noticed that on some videos that I had in .avi format, in which I edited together in Movie Maker, have some what I would call artifact issues or pixelation. In heavy blacks you can see little squares which is annoying. I don't really see it in the original file. Is there a way to fix this? Is it a result of Movie Maker turning the complete project as an mp4 file?

[newpball]

Is there a way to fix this?

Absolutely!

It is actually very easy: use a good codec with a high bitrate!

[Coffindan]

does this work with an .avi file that I didn't make myself?

[newpball]

does this work with an .avi file that I didn't make myself?

Well, listen if the file is bad already you obviously can't make it any better.

You can't make anything good out of spoiled milk.

[Coffindan]

LOL Ok gotcha! So any artifacting or whatever you want to call it that is in your source material I can either a) deal with it or b) move on to a better source. I think the high bit rate encoding is more applicable if I am taking from say an analog source and converting it to a digital format. I'm new to this stuff and trying to learn as much as I can.

[newpball]

LOL Ok gotcha! So any artifacting or whatever you want to call it that is in your source material I can either a) deal with it or b) move on to a better source.

Basically yes.

Sure you can try to cover things up by darkening things, smoothing things, perhaps adding noise but the bottom line is that the quality of your source will go down.

The only exception are cartoons, where post processing can certainly 'improve' things given the assumption that surfaces are more or less detailless.

[Coffindan]

Great info thanks! Any software recommendations for these types of editing (darken, smooth etc.)?

[newpball]

Why not post a small 5 second sample of the original AVI file so we can see what artifacts you are talking about.

[Coffindan]

Watch the clip full screen and you can see the squares in the deep blacks and gray areas. This avi is from a PAL DVD so Im also not sure if that has anything to do with it. I didn't rip it myself.
https://www.youtube.com/watch?v=mLpwBZFDO6c&feature=youtu.be

This scene looked worse on my LED 46" Flatscreen. The .avi file was processed through Handbrake and I inserted an english subtitle .srt file when I converted it from .avi to mp4. I then edited it a little in Windows Movie Maker before saving the project so that I could burn it to a DVDR.

Last week I ripped some trailers from a couple of DVDs (NTSC), cobbled them together in Windows Movie Maker, saved the project under the recommended settings and burn it to a DVDR. It looked great with no problems at all.

[jagabo]

You can use a deblocking filter to reduce blocky artifacts caused by over compression. For example:

http://avisynth.nl/index.php/Deblock_QED

Some decoders have deblocking filters built in. Like Xvid and all h.264 decoders. Of course, these filters can't restore missing detail, just smooth out the blocks.

There are also filters that can reduce posterization artifacts, debanding filters, like GradFun3 in AviSynth.

[LMotlow]

Thanks for the smple, but....

1. We don't spend much time analyzing what uTube does to a video. Where is your original?

2.

some videos that I had in .avi format

"avi" is not a format. It's a container that can be encoded using many different codecs.


3.

which I edited together in Movie Maker

That was your first quality loss. Suggestion: learn to use better encoding software. Likely WMM is where it started.

We need a sample of your original .avi. If you don't know how to make an unprocessed sample from an original, just ask. If you don't know how your original is encoded, use MediaInfo to copy the "Text" view from its report and post it in this thread.

[Coffindan]

Ah yes .avi is not a format. Sorry about that.

OK I downloaded MediaInfo (thanks for the link!). Here is a copy and paste of it's findings.

General
Complete name : C:\DANIEL\celluloid coffin\Il Demonio - Test.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 11.6 MiB
Duration : 51s 576ms
Overall bit rate : 1 881 Kbps
Writing application : Lavf54.63.104

Video
ID : 0
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 1 frame
Codec ID : H264
Duration : 51s 560ms
Bit rate : 1 608 Kbps
Width : 688 pixels
Height : 384 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.243
Stream size : 9.90 MiB (86%)
Writing library : x264 core 129
Encoding settings : cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x1:0x131 / me=dia / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=2 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=abr / mbtree=1 / bitrate=1608 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.41 / aq=1:1.00

Audio
ID : 1
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 3
Mode : Joint stereo
Mode extension : MS Stereo
Codec ID : 55
Codec ID/Hint : MP3
Duration : 51s 576ms
Bit rate mode : Constant
Bit rate : 256 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Compression mode : Lossy
Stream size : 1.57 MiB (14%)
Alignment : Aligned on interleaves
Interleave, duration : 24 ms (0.60 video frame)

The clip is attached.

[LMotlow]

Thanks for the sample. If that .avi is indeed your original, you have quite a job ahead.

In heavy blacks you can see little squares which is annoying. I don't really see it in the original file.

I see plenty of it, and other problems. The dark details are badly crushed. In normal speak, crushed means destroyed. What's left of former detail is just noise -- macroblocks, mottling, smudged details, grime, lots of banding. Crushed darks during processing are made worse by the absurdly low bitrate used, which also gives you edge twitter, motion shimmer, object shift during motion, and lots of other horrible stuff.

The YUV histogram below shows that darks are cut off below a point to the left of the white spike in the image -- there is no detail to the left of that spike.


How this translates into dark-level noise and banding is shown in the images below, frames 621 and 781. If you can't see that during normal work, I brightened everything below RGB 64 to make it more visible here:




Consider it pretty well impossible to clean this garbage without destroying something. You can't restore detail that doesn't exist. You'll need Avisynth for this, and the effort probably won't get you much.

Is there a way to fix this? Is it a result of Movie Maker turning the complete project as an mp4 file?

Consider it pretty well impossible to clean this garbage without destroying something. You can't restore detail that doesn't exist. You'll need Avisynth for this, and the effort might not profit by much. By the time you smooth out this stuff, a lot of former "detail" (which is mostly just noise) turns to mush -- because, in fact, that's what it is.
More detail needs more bitrate.
Shadows and large smooth areas need more bitrate.
motion needs more bitrate.
That's the way this stuff works.

[Coffindan]

I definitely retract what I said about not seeing it in the original. I didn't have it full screen on the pc at the time and really noticed the problem once I burned the dvdr and was watching it on the tv. I get what you are saying though by looking at the stills, I can see that the end result using Avisynth might end up just mushing things up. I downloaded avisynth though and once I figure out how that works, I will experiment. I guess the original rip was done with a low bitrate which caused all this to begin with?

[LMotlow]

I definitely retract what I said about not seeing it in the original. I didn't have it full screen on the pc at the time and really noticed the problem once I burned the dvdr and was watching it on the tv. I get what you are saying though by looking at the stills, I can see that the end result using Avisynth might end up just mushing things up. I downloaded avisynth though and once I figure out how that works, I will experiment. I guess the original rip was done with a low bitrate which caused all this to begin with?

A rip wouldn't cause those problems. A rip is a 1:1 copy or decryption, not a recording, not a capture, not a re-encode. If that rip was really a 1:1 copy -- which is what a rip is -- then that copy would have whatever problems were on the original DVD.

Avisynth didn't quite mush it up. It smoothed the disturbances, or hopefully the worst of it. If you smooth noise and remove some of it, and if most of the "detail" is noise to begin with, you see what's left. The histogram tells me that if those black levels weren't smashed on the original, then they were screwed up during whatever processing followed.

The script I used, below. Others will have different ideas. Encoded with TMPGenc Mastering Works.

SmoothLevels(16, 1.0, 255, 16, 255, chroma=200,limiter=0,tvrange=true,dither=100)
dfttest(sigma=3)
Dither_convert_8_to_16()
GradFun3(lsb_in=true,lsb=true)
DitherPost()
LimitedSharpenFaster(strength=50)
addgrainC(1.5,1.5)
grayscale()

[newpball]

the original DVD.

That's going to be a challenge with this movie.

[LMotlow]

....which leads to some questions. This wasn't a retail DVD, right? If retail PAL, it would have been interlaced (or telecined somehow, since this original was film). BTW, if the original was film, how is this h264 .avi playing at 25fps progressive? Either the original DVD is a total mess, or somewhere between MPEG2 and h264/avi, the worst was done. If the original was truly "PAL DVD", the rip would be MPEG2 and wouldn't be an oddball frame size.

[Coffindan]

I dont know if it was retail or not. I got it from a friend who downloaded it. He didn't mess with. Now I did do a PAL to NTSC conversion when I added the subtitles through Handbrake but the sample video is straight from the original avi file.

[Coffindan]

Im playing around with AvsPmod so I am trying to understand how all this works. This method of editing is very interesting! Now I made that sample with the original avi file with Freemake video converter, so it's possible Freemake might have done something to it but I loaded the file, took out what I didnt want and then converted to avi. Maybe I can figure out how to make another sample with Virtual Dub or something.

[newpball]

Now I did do a PAL to NTSC conversion

Why? You will make a bad thing worse.

The whole movie is on Youtube by the way.

https://www.youtube.com/watch?v=_eyMnR4EdgE

This would be a great one for Criterion to pick up.

Rondi was not only a good director he also was a great screenwriter, "8 1/2", "Giulietta Degli Spiriti" and "La Dolce Vita" are from his hands.

[Coffindan]

YouTube is where I first discovered it. If the original avi is PAL, I would have to convert it in order to view it on a ntsc DVD or blu player right?

[Coffindan]

I just used Mediafile to examine the original avi file and here are the results.

General
Complete name : C:\Daniel\MOVIES\Il Demonio\Il Demonio - The Demon (DivX)\Il Demonio - The Demon (Brunello Rondi 1963) DVD-Rip by davide466.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 1.37 GiB
Duration : 1h 34mn
Overall bit rate : 2 074 Kbps
Writing application : FairUse Wizard - http://fairusewizard.com
Writing library : The best and REALLY easy backup tool

Video
ID : 0
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5
Format settings, BVOP : 1
Format settings, QPel : No
Format settings, GMC : No warppoints
Format settings, Matrix : Default (MPEG)
Muxing mode : Packed bitstream
Codec ID : XVID
Codec ID/Hint : XviD
Duration : 1h 34mn
Bit rate : 1 936 Kbps
Width : 688 pixels
Height : 384 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.293
Stream size : 1.28 GiB (93%)
Writing library : XviD 1.2.0.dev47 (UTC 2006-11-01)

Audio
ID : 1
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 3
Mode : Joint stereo
Mode extension : MS Stereo
Codec ID : 55
Codec ID/Hint : MP3
Duration : 1h 34mn
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Compression mode : Lossy
Stream size : 86.3 MiB (6%)
Alignment : Aligned on interleaves
Interleave, duration : 40 ms (1.00 video frame)
Interleave, preload duration : 504 ms
Writing library : LAME3.97
Encoding settings : -m j -V 4 -q 2 -lowpass 17 -b 128

[manono]

If the original avi is PAL...

Avis are neither PAL nor NTSC. Your 'AVI' is 25fps so it came from a PAL source.

...I would have to convert it in order to view it on a ntsc DVD or blu player right?

Yes, most likely. AvsToDVD can do the conversion while making a DVD for you. Ignore newpball.

Now I made that sample with the original avi file with Freemake video converter, so it's possible Freemake might have done something to it but I loaded the file, took out what I didnt want and then converted to avi.

Are you saying you already reencoded the whole thing, as well as the sample? You didn't get this thing as an AVI? As far as I know Freemake isn't just a cutter and doesn't just repackage videos into different containers.

[LMotlow]

YouTube is where I first discovered it. If the original avi is PAL, I would have to convert it in order to view it on a ntsc DVD or blu player right?

Like manono says, this avi is neither PAL nor NTSC frame size or encoding, just a PAL frame rate. The GOP size of the .avi is not valid for any flavor of DVD.

This gets to be more interesting. So this "DVD" isn't a DVD at all, it's a DVD disc with an .avi video burned to it as "data. It's a movie, which I already knew, but why is it playing at 25fps progressive? Nobody shoots major-studio film at 25 fps. And no DVD, whether PAL or not, is ever 688x384. So this is not a DVD rip, it's formerly a digitized 23.976fps wide screen movie ripped-off from a 16:9 DAR letterboxed DVD and converted to trash by a low-rent Really Cool Moron with no respect for the source. Okay, then this avi-that-isn't-a-DVD really is your only source. I wouldn't try too much of a cleanup -- really, there's not very much you can do with it beyond some anti-banding and smoothing. You'd have to resample audio, because you can't make PAL or NTSC DVD or BluRay with mp3 audio, and certainly not at variable audio rates.

Problem now is figuring out what was done to convert this movie from 23.976 film to 25fps PAL. It's possible that what the Cool Moron did was speed it up to 25fps, in which case you could make it 29.97fps NTSC by resetting the frame rate and audio to 23.976 in Avisynth and soft-coding some pulldown. But I didn't try slowing this to 23.976 or 24fps, and didn't notice dupe frames or pulldown anywhere. It ain't interlaced, but lots of progressive DVD's get encoded with fake interlace flags. Probably a little bit missing from the frame, too: movies in 1963 weren't made at 16:9, this was probably 1.85:1.

[EDIT] So I just read before posting this note that you apparently have a DVD that you re-encoded? Your sample wasn't a piece of the original? Let's make up our mind here. You just posted that you started with an avi and "converted it to avi" ???

Come on. You're burning up a lot of time and effort.

[LMotlow]

I just used Mediafile to examine the original avi file and here are the results.

General
Complete name : C:\Daniel\MOVIES\Il Demonio\Il Demonio - The Demon (DivX)\Il Demonio - The Demon (Brunello Rondi 1963) DVD-Rip by davide466.avi........

I'm afraid you're confusing us. Please stop referring to this "original" video as a DVD rip. It's not an original and it's not a rip. It's a re-encode. And re-encoded to a very lossy, obsolete codec that botched the DVD source for good. I'm afraid that's what you'll have to work with. Unless you can find a decent copy of that movie using a better codec.

You should probably ignore the script I posted. It was for an AVC/h264 re-encode. Whether it will work with your "original" is up for debate.

[jagabo]

I just used Mediafile to examine the original avi file and here are the results.

General
Complete name : C:\Daniel\MOVIES\Il Demonio\Il Demonio - The Demon (DivX)\Il Demonio - The Demon (Brunello Rondi 1963) DVD-Rip by davide466.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 1.37 GiB
Duration : 1h 34mn
Overall bit rate : 2 074 Kbps
Writing application : FairUse Wizard - http://fairusewizard.com
Writing library : The best and REALLY easy backup tool

Video
ID : 0
Format : MPEG-4 Visual
Format profile : Advanced Simple@L5

If your source was an XVID AVI why did you upload an x264 re-encoded sample? What a waste of time. Use VirtualDub in Video -> Direct Stream Copy mode to trim out a short sample for upload.

[Coffindan]

I definitely retract what I said about not seeing it in the original. I didn't have it full screen on the pc at the time and really noticed the problem once I burned the dvdr and was watching it on the tv. I get what you are saying though by looking at the stills, I can see that the end result using Avisynth might end up just mushing things up. I downloaded avisynth though and once I figure out how that works, I will experiment. I guess the original rip was done with a low bitrate which caused all this to begin with?

A rip wouldn't cause those problems. A rip is a 1:1 copy or decryption, not a recording, not a capture, not a re-encode. If that rip was really a 1:1 copy -- which is what a rip is -- then that copy would have whatever problems were on the original DVD.

Avisynth didn't quite mush it up. It smoothed the disturbances, or hopefully the worst of it. If you smooth noise and remove some of it, and if most of the "detail" is noise to begin with, you see what's left. The histogram tells me that if those black levels weren't smashed on the original, then they were screwed up during whatever processing followed.

The script I used, below. Others will have different ideas. Encoded with TMPGenc Mastering Works.

SmoothLevels(16, 1.0, 255, 16, 255, chroma=200,limiter=0,tvrange=true,dither=100)
dfttest(sigma=3)
Dither_convert_8_to_16()
GradFun3(lsb_in=true,lsb=true)
DitherPost()
LimitedSharpenFaster(strength=50)
addgrainC(1.5,1.5)
grayscale()

Wow! Taking a look at the clip and I can really tell a difference! I didn't look at it last night because the download was going to take awhile. Here at work our speed is really fast so it only took a few seconds.

[Coffindan]

YouTube is where I first discovered it. If the original avi is PAL, I would have to convert it in order to view it on a ntsc DVD or blu player right?

Like manono says, this avi is neither PAL nor NTSC frame size or encoding, just a PAL frame rate. The GOP size of the .avi is not valid for any flavor of DVD.

This gets to be more interesting. So this "DVD" isn't a DVD at all, it's a DVD disc with an .avi video burned to it as "data. It's a movie, which I already knew, but why is it playing at 25fps progressive? Nobody shoots major-studio film at 25 fps. And no DVD, whether PAL or not, is ever 688x384. So this is not a DVD rip, it's formerly a digitized 23.976fps wide screen movie ripped-off from a 16:9 DAR letterboxed DVD and converted to trash by a low-rent Really Cool Moron with no respect for the source. Okay, then this avi-that-isn't-a-DVD really is your only source. I wouldn't try too much of a cleanup -- really, there's not very much you can do with it beyond some anti-banding and smoothing. You'd have to resample audio, because you can't make PAL or NTSC DVD or BluRay with mp3 audio, and certainly not at variable audio rates.

Problem now is figuring out what was done to convert this movie from 23.976 film to 25fps PAL. It's possible that what the Cool Moron did was speed it up to 25fps, in which case you could make it 29.97fps NTSC by resetting the frame rate and audio to 23.976 in Avisynth and soft-coding some pulldown. But I didn't try slowing this to 23.976 or 24fps, and didn't notice dupe frames or pulldown anywhere. It ain't interlaced, but lots of progressive DVD's get encoded with fake interlace flags. Probably a little bit missing from the frame, too: movies in 1963 weren't made at 16:9, this was probably 1.85:1.

[EDIT] So I just read before posting this note that you apparently have a DVD that you re-encoded? Your sample wasn't a piece of the original? Let's make up our mind here. You just posted that you started with an avi and "converted it to avi" ???

Come on. You're burning up a lot of time and effort.

I apologize for any confusion and lack of knowledge on my part. I am totally new to a majority of this stuff. I made another sample this morning by using the original avi file. I used Virtual Dub to make a clip almost the same as the first one I posted. I haven't uploaded it yet though.

And yeah using freemake to make a clip and then choosing to convert an avi to an avi was pretty dumb. I wasn't thinking when I did that. At least with Virtual Dub I was able to make a clip and save it without anything extra being done to the clip. Like I said, I am new to this.

I guess what I want to do is take the movie, clean it up as best I can, burn it to a DVD with a nice little menu and have it for my collection.

[LMotlow]

Okay, we all had to start somewhere and get this stuff piecemeal. The problem with apps like WMM and similar is that you don't have to know what you're doing, so you know next to nothing when you see the results. One of the first things you pick up is that filtering designed for one video won't work so well with something else.

I keep reading your post earlier where you say the person who made this DVD didn't mess with it. Well....I guess you know by now that this isn't true. You can make a DVD from it, but there are structural problems to deal with if the MediaInfo on the "original" is anything to go by. Got a feeling the avi will be a problem child, but we'll see what can be done. Looking forward to an unprocessed sample. If you have problems making it, let us know.

[Coffindan]

I keep reading your post earlier where you say the person who made this DVD didn't mess with it.

Yeah I guess I really don't know that. That avi was downloaded at some point and then given to me. So whoever uploaded it to begin with is responsible for the structural problems I guess. If it turns out to be a problem child, at least it's something I learn with. And you are on the money about Windows Movie Maker. You don't need to know much of anything to use the software. It was really easy for me to start with but I would like to move on to something else at some point. And I really appreciate everyone's input on this thread. LOTS of info to consider and take in.