Strange lines in ISO

[Saynom]

Hi everyone,

I recently decided to convert to x265 some ISO I have from a DVD box I've lost for quite some times and I have an issue with one of them.

It has a "line overlay" like this (part of the image): http://puu.sh/pmEHl/08723d0660.png

So they're still here when I convert it. It's pretty ugly...

I don't know what went wrong when I saved it in ISO but is there a way to correct this now I don't have the source anymore?

Thanks in advance for your help.

[jagabo]

It's hard to say from your sample but it looks like you need a proper deinterlace or inverse telecine.

[Saynom]

It's hard to say from your sample but it looks like you need a proper deinterlace or inverse telecine.

Thanks for your reply. How could I achieve this the easiest way?

Also, here is a full screenshot: http://puu.sh/pmRuV/d6c6a169e2.png

[manono]

The first thing you do is provide a short 10 second sample from your DVD, one with steady movement.

[Saynom]

The first thing you do is provide a short 10 second sample from your DVD, one with steady movement.

Here you go! (from the ISO)

[jagabo]

We need a sample of the original 720x576 DVD in MPEG 2, not a reencoded video. But my guess is that the analog tape deck that was used to make the DVD was delivering one field slightly brighter than the other. You can fix this by adjusting the brightness of one field to match the other, or just using a mild vertical blur filter to smooth them out.

[LigH.de]

I know such slight lines already from interlaced interpretation of chrominance in scenes which would have progressive content, but are encoded in interlaced mode just to be safe on a DVD. Such material can be hard to handle due to a lack of certainty...

Just compare the results of ConvertToYUY2(interlaced=true) and ConvertToYUY2(interlaced=false) right after MPEG2Source in AviSynth; you may see differences in the preview. Just for watching, not for converting.

[Saynom]

We need a sample of the original 720x576 DVD in MPEG 2, not a reencoded video. But my guess is that the analog tape deck that was used to make the DVD was delivering one field slightly brighter than the other. You can fix this by adjusting the brightness of one field to match the other, or just using a mild vertical blur filter to smooth them out.

Here you go. (Sorry to be late)

How can I do what you recommended? Using which software?

I know such slight lines already from interlaced interpretation of chrominance in scenes which would have progressive content, but are encoded in interlaced mode just to be safe on a DVD. Such material can be hard to handle due to a lack of certainty...

Just compare the results of ConvertToYUY2(interlaced=true) and ConvertToYUY2(interlaced=false) right after MPEG2Source in AviSynth; you may see differences in the preview. Just for watching, not for converting.

Wow, that turned out to be pretty harsh since I didn't know how to use AviSynth... But I eventually managed to try it and it doesn't seems to have any impact...

[LigH.de]

I didn't mean to be harsh, sorry if you took it that way; I only wanted to document my experience in one case to others who would understand what I mean.

Those lines are in the source, and they are static regarding their position (always the same lines are brighter or darker than the average). If you look at a bobbed result, they make the video flicker a lot but look rather soft when looking at single fields, like typical artefacts of interlaced resizing.

My guess is that they are the result of a field-related height scaling where it was not necessary. The video studio produced waste. Not sure how to fix that easily, you would possibly need a field based frequency correction; AviSynth might provide that with some plugin like Fizick's DeFreq, but it's not easy to use.

[Saynom]

I didn't mean to be harsh, sorry if you took it that way; I only wanted to document my experience in one case to others who would understand what I mean.

Those lines are in the source, and they are static regarding their position (always the same lines are brighter or darker than the average). If you look at a bobbed result, they make the video flicker a lot but look rather soft when looking at single fields, like typical artefacts of interlaced resizing.

My guess is that they are the result of a field-related height scaling where it was not necessary. The video studio produced waste. Not sure how to fix that easily, you would possibly need a field based frequency correction; AviSynth might provide that with some plugin like Fizick's DeFreq, but it's not easy to use.

Ahah mate, I didn't mean you were harsh, I meant that using AviSynth for the first was an harsh task. Sorry for the misunderstanding, I'm struggling to find the right word in english sometimes. No worries!

As for your explanation, darn it... That's quite frustrating. I really was hoping to find a way to fix that.

Anyway, thanks for your help!

[LigH.de]

Well, if you are lucky, someone with experience might post a script. But I fear I am not the one with so much experience regarding how to use DeFreq or something similar. But I could point other people here...

[Saynom]

Well, if you are lucky, someone with experience might post a script. But I fear I am not the one with so much experience regarding how to use DeFreq or something similar. But I could point other people here...

That would be very nice of you. Thanks.

EDIT: Also, why, after encoding my videos, is there like an "halo" of pixels from the background moving with/around people's head?

[jagabo]

The vinverse filter in AviSynth will get rid of most of it without otherwise damaging the video much. It doesn't come with AviSynth. You'll need to download and install it.

http://avisynth.nl/index.php/Vinverse

If you're going to learn AviSynth you might as well do noise reduction and upscaling with it too.

[jagabo]

Also, why, after encoding my videos, is there like an "halo" of pixels from the background moving with/around people's head?

Bad encoding in the source. Exacerbated by sharp upscaling and reencoding.

[LigH.de]

I doubt that Vinverse will be the correct countermeasure here. On one hand, the issue is mainly in the chrominance planes; on the other hand, DeFreq analysis reveals that there are constant frequency distortions, therefore the best method will be discovering the distortion frequency and filter it out. I just did not yet understand how...

[jagabo]

vinverse can be used on the chroma planes too with:

YtoUV(UtoY().Vinverse, VtoY().vinverse(), vinverse())

Defreq may work a little better but I suspect the OP will find this sufficient.

[Saynom]

vinverse can be used on the chroma planes too with:

YtoUV(UtoY().Vinverse, VtoY().vinverse(), vinverse())

Defreq may work a little better but I suspect the OP will find this sufficient.

Wow! It seems to do the trick (more or less)! Thanks a lot. How do I do for using this script with my file before encoding in x265 with handbrake? Sorry, I'm so newbie...

And about

If you're going to learn AviSynth you might as well do noise reduction and upscaling with it too.

Why shoul I upscale? What script would you recommend for noise reduction?

Bad encoding in the source. Exacerbated by sharp upscaling and reencoding.

Is there something I could do about that? I've already set Handbrake on a good quality, I don't know if I can do better without having files that are too big in the end.

Also, I would have an other question. I want to join files with mkvtoolnix but some of them are 718x576 instead of 720x576 so it won't mux them. I tried a lot of things, including ffmpeg but I can't find a way to resize without re-encode. And mkvtoolnix is pretty picky when in comes to details... How should I do?

[jagabo]

How do I do for using this script with my file before encoding in x265 with handbrake?

Handbrake can't work directly with AviSynth scripts. I've never used it but I've heard of people using AVFS. I use the command line version of x264 to do all my encoding. That accepts AviSynth scripts directly.

Why shoul I upscale?

The MP4 file you uploaded was upscaled so I assumed that's what you were planning. I'd use nnedi3 for upscaling. And usually a little sharpening before or after.

What script would you recommend for noise reduction?

There are many noise reduction filters. These days I mostly use TemporalDegrain(). But there's also MCTemporalDenoise(), dfttest(), TTempSmooth(), etc.

Bad encoding in the source. Exacerbated by sharp upscaling and reencoding.

Is there something I could do about that?

Mpeg2Source() has deringing filters that can be applied. That will remove some of the DCT noise but won't help with the larger distortions. Some of the general noise reduction filters might help a little.

Also, I would have an other question. I want to join files with mkvtoolnix but some of them are 718x576 instead of 720x576 so it won't mux them. I tried a lot of things, including ffmpeg but I can't find a way to resize without re-encode. And mkvtoolnix is pretty picky when in comes to details... How should I do?

You can't join files of different resolution without reencoding. If all your sources are DVD there shouldn't be any that at 718 pixels wide. What exactly are you joining?

Here's a sample script that shows some of the above filters in use:

Mpeg2Source("test lines dvd quality.d2v")
YtoUV(UtoY().Vinverse, VtoY().vinverse(), vinverse()) # reduce horizontal lines
ColorMatrix(mode="rec.601->rec.709") # HD should use rec.709
TemporalDegrain(SAD1=100, SAD2=75, sigma=4) # mild noise reduction
Sharpen(0.3)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1280, fheight=720) # upscale to 720p
Sharpen(0.3)

Tune parameters to your liking. And for particular sources.

[Saynom]

Handbrake can't work directly with AviSynth scripts. I've never used it but I've heard of people using AVFS. I use the command line version of x264 to do all my encoding. That accepts AviSynth scripts directly.

You mean you do that with ffmpeg? If so, it would be awesome if you could give me the script you use(I'm still new to it). I usually convert with "Container: MKV" "Picture: Anamorphic: strict", "Filters: decomb: default", "video codec: H.265", "FPS: Same as source, constant framerate", "x265 preset: medium", "quality: constant quality, 22", "audio: aac, 192k", and chapters are set automatically along the ISO timings. Thanks a lot in advance.

The MP4 file you uploaded was upscaled so I assumed that's what you were planning. I'd use nnedi3 for upscaling. And usually a little sharpening before or after.

Well, it was just upscale because I didn't know how to keep only 10sec so I used vegas and it didn't turn very well. I'm pretty fine with the original dvd resolution (talking about that isn't it strange that, even if it's in 4:3 when I read them in VLC I can go on full screen and it fills all my screen!?). Is there any advantage to upscaling?

There are many noise reduction filters. These days I mostly use TemporalDegrain(). But there's also MCTemporalDenoise(), dfttest(), TTempSmooth(), etc.

I will try that with the script you leave me at the bottom.

Mpeg2Source() has deringing filters that can be applied. That will remove some of the DCT noise but won't help with the larger distortions. Some of the general noise reduction filters might help a little.

Hmm okay, worth a try.

You can't join files of different resolution without reencoding. If all your sources are DVD there shouldn't be any that at 718 pixels wide. What exactly are you joining?

Darn it... But handbrake doesn't allow me to set resolution to 720x576 when the source is 718x576(even in custom mode). I can't find a way to do that...
And yea, I was surprised as well but almost all the "sub-files/chapters" in the ISOs are a bit different (some are 704x576, etc, even inside the same DVD), I don't understand why. So as handbrake encode them separately, I would want to join them after. But I can't because the resolutions of these DVDs are absurd...

Here's a sample script that shows some of the above filters in use:

Mpeg2Source("test lines dvd quality.d2v")
YtoUV(UtoY().Vinverse, VtoY().vinverse(), vinverse()) # reduce horizontal lines
ColorMatrix(mode="rec.601->rec.709") # HD should use rec.709
TemporalDegrain(SAD1=100, SAD2=75, sigma=4) # mild noise reduction
Sharpen(0.3)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=1280, fheight=720) # upscale to 720p
Sharpen(0.3)

Tune parameters to your liking. And for particular sources.

Thanks mate, I really appreciate your help!

[LigH.de]

There are definitely no DVDs with 718 pixels width. You could only end up with such unusual resolutions (not even a multiple of 16, which is the size of a "macroblock", the encoding unit of MPEG-2 videos) if some moron tried to crop every littlest remains of black borders. But then it is not a DVD anymore, the "DVD Video" specifications allow only few distinct frame dimensions and aspect ratios. 704 and 720 pixels are allowed as width. 718 pixels are not possible on a DVD Video. Maybe in a copy with cropped corders, but still, it would be not optimal to only crop such a little rest.

Handbrake is based on ffmpeg (or: libav codecs and formats) to be portable across different operating systems. AviSynth is a Windows-only frameserver, though. Therefore Handbrake may not contain routines which care about AviSynth and the VfW API.

x264 is a separate CLI encoder. ffmpeg may contain libx264, and there are also modified x264 encoders containing some decoders from ffmpeg (libav); but basically, x264 could also exist without any relation to ffmpeg. Windows builds of x264 also usually contain routines to allow AviSynth input. x265 is also a separate CLI encoder, but does not yet contain AviSynth input (maybe except for modified builds); you would have to use a helper tool to pipe AviSynth output into x265 (e.g. avs4x26x or MuldeR's Simple x264/x265 Launcher or StaxRip or MeGUI or ...).

There are also many Windows builds of ffmpeg which can use AviSynth as input. If you also load the audio stream in the same script (e.g. with Nic's audio source filter, or FFMS2, or L-SMASH Works ...), combining both using AudioDub(), you can use ffmpeg with libx264 to encode the video stream to AVC or libx265 to encode the video stream to HEVC, and ffmpeg's AAC encoder to encode the audio stream to LC AAC, multiplexed right into a container supporting both (MP4 or MKV). Unfortunately, building a command line for you is beyond my experience. But there are some GUIs for ffmpeg which may help you doing so, like Selur's Hybrid, or TEncoder, and probably half a dozen more.

[Saynom]

There are definitely no DVDs with 718 pixels width. You could only end up with such unusual resolutions (not even a multiple of 16, which is the size of a "macroblock", the encoding unit of MPEG-2 videos) if some moron tried to crop every littlest remains of black borders. But then it is not a DVD anymore, the "DVD Video" specifications allow only few distinct frame dimensions and aspect ratios. 704 and 720 pixels are allowed as width. 718 pixels are not possible on a DVD Video. Maybe in a copy with cropped corders, but still, it would be not optimal to only crop such a little rest.

I wish you were right... When I put my ISOs in handbrake (which is set up to "strict" resolution, so that's the real resolution of DVDs, right?), I get all these resolutions on output files:
720x576, 718x576, 702x576, 708x576, 710x576, 714x572, 712x572, 716x572, 714x558, 716x558.

That's just beyond comprehension...

Handbrake is based on ffmpeg (or: libav codecs and formats) to be portable across different operating systems. AviSynth is a Windows-only frameserver, though. Therefore Handbrake may not contain routines which care about AviSynth and the VfW API.

x264 is a separate CLI encoder. ffmpeg may contain libx264, and there are also modified x264 encoders containing some decoders from ffmpeg (libav); but basically, x264 could also exist without any relation to ffmpeg. Windows builds of x264 also usually contain routines to allow AviSynth input. x265 is also a separate CLI encoder, but does not yet contain AviSynth input (maybe except for modified builds); you would have to use a helper tool to pipe AviSynth output into x265 (e.g. avs4x26x or MuldeR's Simple x264/x265 Launcher or StaxRip or MeGUI or ...).

There are also many Windows builds of ffmpeg which can use AviSynth as input. If you also load the audio stream in the same script (e.g. with Nic's audio source filter, or FFMS2, or L-SMASH Works ...), combining both using AudioDub(), you can use ffmpeg with libx264 to encode the video stream to AVC or libx265 to encode the video stream to HEVC, and ffmpeg's AAC encoder to encode the audio stream to LC AAC, multiplexed right into a container supporting both (MP4 or MKV). Unfortunately, building a command line for you is beyond my experience.

Wow, thanks for these explanations. I don't know if I fully understand all the whys and wherefores because it sounds a bit like an Heath Robinson way to encode to me (you're far more advance on this topic than me), but I appreciate you took the time to make me understand that it would not be as easy as I thought...

But there are some GUIs for ffmpeg which may help you doing so, like Selur's Hybrid, or TEncoder, and probably half a dozen more.

This is quite a good news! I've just downloaded Selur's Hybrid, which looks pretty nice, and I will try it now. Thanks for these references and for your help.

[LigH.de]

When I put my ISOs in handbrake (which is set up to "strict" resolution, so that's the real resolution of DVDs, right?), I get all these resolutions on output files:
720x576, 718x576, 702x576, 708x576, 710x576, 714x572, 712x572, 716x572, 714x558, 716x558.

Possibly not right. The real resolution of DVD Video content is verbosely described in their specs. But it will include possible black borders around the "interesting area" of the video.

I don't know Handbrake, I prefer AviSynth based converters like MeGUI, or even manual preparation ... so I don't know what "strict" means here, but I fear it tries to detect every bit of black border and crop it off, keeping only the "interesting area", which has those variable resolutions. That may not be too bad if you will scale the result by some significant amount, and independently of other movies. But if you want to mainly keep a common scaling factor and the same aspect ratio between several movies, then this would cause issues, as you see.

Not using Handbrake myself, I can't tell you in detail how to set it up to avoid finicky cropping.

[Saynom]

When I put my ISOs in handbrake (which is set up to "strict" resolution, so that's the real resolution of DVDs, right?), I get all these resolutions on output files:
720x576, 718x576, 702x576, 708x576, 710x576, 714x572, 712x572, 716x572, 714x558, 716x558.

Possibly not right. The real resolution of DVD Video content is verbosely described in their specs. But it will include possible black borders around the "interesting area" of the video.

I don't know Handbrake, I prefer AviSynth based converters like MeGUI, or even manual preparation ... so I don't know what "strict" means here, but I fear it tries to detect every bit of black border and crop it off, keeping only the "interesting area", which has those variable resolutions. That may not be too bad if you will scale the result by some significant amount, and independently of other movies. But if you want to mainly keep a common scaling factor and the same aspect ratio between several movies, then this would cause issues, as you see.

Not using Handbrake myself, I can't tell you in detail how to set it up to avoid finicky cropping.

You may be right. That's the only reasonable explanation.

Unfortunately I can't use MeGUI since it doesn't seem to support x265, as far as I know.

No problem about you not using handbrake, I can understand.
I'm trying Hybrid right now and I will let you know how it turns.

EDIT: Damn, it's so slow...

[LigH.de]

MeGUI does support x265, but just basically (preset + tuning, not many specific options). And you may need to set it to the development updater, not the stable one (which is often not really up-to-date).

[jagabo]

"Strict" in Handbrake refers to the aspect ratio, not the frame size. It still crops away black borders. Set cropping to "none" to retain the DVD's original frame size. That's almost always 720x576 but it could be 704x576, 352x576, or 352x288. No other frame size is valid for PAL DVD.

Personally, I don't encode as h.265/HEVC yet. None of my players support it (except the computers, of course). I don't need to reduce every file as small as possible. I just want reasonably small files with good quality. x264 is good enough for me now.

[Saynom]

MeGUI does support x265, but just basically (preset + tuning, not many specific options). And you may need to set it to the development updater, not the stable one (which is often not really up-to-date).

Oh, I didn't see that. I will take a look if I can't get what I want from Hybrid

"Strict" in Handbrake refers to the aspect ratio, not the frame size. It still crops away black borders. Set cropping to "none" to retain the DVD's original frame size. That's almost always 720x576 but it could be 704x576, 352x576, or 352x288. No other frame size is valid for PAL DVD.

Personally, I don't encode as h.265/HEVC yet. None of my players support it (except the computers, of course). I don't need to reduce every file as small as possible. I just want reasonably small files with good quality. x264 is good enough for me now.

Wow, I feel so stupid right now... That's exactly what I should have done... But now I have a good part of my files already encoded. Will it cause troubles if I keep them with these ratios?

And you're right, it's not fully supported on every devices for now but I only have a PC for watching stuff so that's ok for me. I hope it will be widely supported everywhere soon though.

[Saynom]

I've made several tests but I can't find a way to have a dvd resolution which adapt to fill the screen.

When I switch in full screen I get something like that: http://puu.sh/ppMTj/82577cfde2.png

Also the quality is meh. Any idea to fix this?

[jagabo]

When the DVD aspect ratio doesn't match the content aspect ratio the DVD will be padded with black borders. What you have is a 4:3 DVD with a 16:9 content. So the top and bottom of the frame are padded with black letterbox bars. Since you didn't crop those black bars they are part of your encoded video. The player doesn't know they aren't really part of the picture so enlarges the height of the image to touch the top and bottom of the screen, then adds pillarbox bars to the left and right edges to fill the screen. In your video the letterbox bars aren't pure black so you can see the difference between the letterbox bars that are part of the video and the pillarbox bars the player added. The exact same thing will happen when you play the original disc on a DVD player and watch on TV.

Most players have a zoom feature you can use to eliminate the borders. Or you can crop the black borders before encoding. For 16:9 content on a 4:3 PAL DVD you want to remove about 72 lines off the top and 72 lines off the bottom. Sometimes the borders aren't equal so you may have to adjust that. For example, 68 off the top, 76 off the bottom, but still a total of 144 lines.