Reducing moire on DVD rips using Handbrake

[ministry88]

Hello!

I've started ripping my library of DVDs using Handbrake so I can upload them to Plex for streaming on my iPad and Apple TV (4th Gen). My first rip went quite well, but I noticed moire on the pattern of a character's suit (see screenshot -- it doesn't show in a still shot, but just to give you an idea of the shimmer I'm seeing). After futzing around in various forums, it seems either leaving de-interlacing off entirely or setting it to the "decomb" option may be the the way to go. I've read that leaving it on "decomb" may be best to account for some DVDs that are interlaced (most are progressive, if I understand correctly). However, my rip was done with the "decomb" option on, and it caused that moire/shimmer. Should I turn deinterlace off then? Or are there certain situations where I'd want to keep it on? How would I diagnose which DVDs need some kind of de-interlacing? I do have some DVD-Rs of old VHS, and I know VHS is interlaced, so should I apply deinterlacing for those?

In addition to solving the moire problem, I would just like to know the best video settings for ripping DVDs with Handbrake. On the Plex forums I found a post with these settings/workflow, and they seem sensible to me:

MP4 container using h.264 video
Remove all non English audio tracks
Remove all subtitles from file
Add AAC 2 channel 256kbps audio as first track (for compatibility with all devices) if needed
Copy all remaining English audio tracks.
Setup MP4 for "fast start" or "web optimized" whichever you wish to call it.

If the file already contains an h.264 video then the file is remuxed but if the video is not already h.264 then the file is transcoded using Handbrake with a HIGH PROFILE and web optimize.

-With the above in mind, I'm planning on using the "HQ480p30Surround" preset for my DVDs with all the video presets untouched (except, possibly, for the deinterlacing presets). Does this sound like an appropriate setting for DVDs? Should I set the framerate to "same as source" to accommodate my PAL DVDs?
-For audio: in addition to adding the AAC 2-channel track as the first track for compatibility, what settings do I need to adjust to ensure that the original Dolby Surround track(s) will play on my receiver? I'm assuming I'll need to include the tracks with the "AC-3 Passthru" option because Apple TV can't natively decode 5.1 like a receiver can. I'm also wondering about "mixdown" settings -- is that for receivers that can't decode full surround tracks? Would a setting of "Dolby Pro Logic II" allow the audio to be played on my Apple TV through my TV speakers? I'm guessing it still will because even though Pro Logic is not a native Apple TV audio format, the codec is AAC which should still allow the audio to play on my TV via my Apple TV. I've included a screenshot of the audio settings to get a better idea of what I'm asking.

Thanks in advance for any help!

[hello_hello]

I don't use Handbrake much myself, however.....

The problem is, DVDs can be an encoding minefield because there's many possibilities. For NTSC, film can be encoded as progressive, it can be encoded using 3:2 pulldown (which looks similar to interlaced video if it's decoded as progressive), or it can contain interlaced video, and sometimes it's a combination of all three.

Handbrake's Decomb filter is designed to automatically deal with that, and I think by default it outputs a variable frame rate, so in theory the film sections with pulldown are detelecined for 23.967fps progressive, the progressive parts are left progressive, the video sections are de-interlaced to 29.97fps, and the output frame rate changes accordingly.

I've seen quite a few Handbrake encodes where someone forced a constant 29.970fps for a "hybrid" NTSC DVD, so the interlaced sections were correctly de-interlaced to 29.970fps, but as the decomb filter converts the film sections to 23.976fps, every fourth frame has to be duplicated to output a constant 29.97fps and that's hard to watch.

It's a big topic and ideally you'd not using any filtering for a fully progressive source, but unless you want to spend a fair bit of time on encoding, the easiest solution is to leave the decomb filter enabled and let Handbrake output a variable frame rate if need be, and if you do want to spend a lot of time on it then you probably wouldn't use Handbrake as it's designed to be fairly automatic.

Are you sure the decomb filter caused the moire/shimmering? It wouldn't be unusual for it to be present in the source video.

The "add AAC 2 channel 256kbps audio as first track" idea is mainly for pandering to Apple devices that don't support AC3 or multichannel audio. The Prologic II option downmixes the audio to stereo, but the surround information is encoded in such a way that it can be decoded as surround again by a Prologic decoder, although it'll never be as good as discreet 5.1ch. A non-Prologic decoder will decode it as normal stereo.

[ministry88]

First, thank you for responding to my bear of a post.

Yes, I've heard that DVDs are a scrambled egg of encodes. Combined with your statement of DVD encoding and my target goal (being able to watch my film and TV library on-the-go, with occasional use in my home theater), I think I'll just go with the Handbrake default settings for "HQ480p30 Surround." In this case, "good enough" is good enough. On-the-go viewing (read: hotel rooms and friends' houses) won't be on a home theater or on a large HDTV so any limitations in the source probably would be small at worst.

Question: So what I'm hearing from you is that a variable framerate would be best, and Handbrake suggests setting it to "same as source" as best practice. Would my PAL DVDs still display properly with a variable framerate setting?

To answer your question about moire: no, it was not present in the original source. Don't really know what to do to remedy that, but I'll try the HQ480p30 encode which is better quality than the rip that displayed the moire.

Thanks again!

[jagabo]

30p is almost always wrong. Almost everything shot on film is 24p.

[poisondeathray]

It should be "detelecine" shouldn't it ? Or did handbrake take that out in recent versions ? It used to be "detelecine" was the same as IVTC

When you use decomb, if it detects "combing" it will apply a deinterlace. Things like striped patterns on clothing "look" a lot like combing. When you deinterlace progressive content , you will get that degredation , aliasing and moire patterns

[ministry88]

Hi poisondeathray (awesome name by the way),

Yes, there is still a detelecine option in the newest handbrake. Would there be any harm in leaving detelecine on at all times for DVDs? It would be in conjunction with the aforementioned framerate settings of "same as source" with a variable bitrate and deinterlace set to "decomb." Would the "decomb" deinterlace setting conflict with the "detelecine" option being activated? Should I use a different deinterlace standard such as "Yadif"? Finally: what preset should I use for deinterlace? My options are "Bob," "Custom," "EEDI2," and "EEDI2Bob"

Thanks for the explanation of combing/decombing.

[poisondeathray]

I dont really use handbrake , but detelecine would only be for film dvd's . Theatrical sources . Hollywood movies. Or TV Dramas. For things like sports DVD's, reality show DVD's - you wouldn't use it because those are usually interlaced content . Detelecine (or inverse telecine) removes the telecine and returns the original progressive film frames

If there is residual combing , decomb should deinterlace. But that' s the problem - any clothing that has fine stripped patterns might get detected as "combed" . AFAIK, there is no manual override you can specify for frame ranges. Handbrake is meant to be "easy to use" . I suspect if you used detelecine, but disabled decomb, it would work on that specific DVD on that specific section. For other generic film DVD's detelecine + decomb probably is the better general use choice

EEDI2 is higher quality than yadif. The double rate bob options are for true interlaced content.

There should be some guides somewhere for handbrake options

[ministry88]

Thanks again! OK, I'll try detelecine on its own without decombing and see if that removes the moire. Since most of my collection is commercial DVDs with only about 10% DVD-R, I'll use your suggestion of using detelecine+decomb, and spot-check/tweak the 45 or so DVD-Rs I have.

Again, handbrake is great for me because it's "easy to use" -- good enough quality is good enough for me. Minus the moire of course

[jagabo]

Thanks again! OK, I'll try detelecine on its own without decombing and see if that removes the moire.

When you detelecine you want to decimate to 23.976 fps too. Though, as I recall, Handbrakes decimation doesn't always work so well.

For interlaced content it's best to decomb and double the frame rate to 59.94 fps (50 fps PAL). That will get you smoother, less flickery playback. Unless your playback device can't handle 59.94 fps (rare on modern devices).

[ministry88]

Would I want to set the 23.976 framerate to "constant framerate" or "peak framerate"? Same question for the 59.94 frame rate.

[jagabo]

Some devices (mostly older ones) can't handle variable frame rate video. To be safe I always stick with constant frame rate. Cartoons/anime sometimes benefit from VFR.

[hello_hello]

Question: So what I'm hearing from you is that a variable framerate would be best, and Handbrake suggests setting it to "same as source" as best practice. Would my PAL DVDs still display properly with a variable framerate setting?

I'm using an older version of Handbrake (the last that'll run on XP, but assuming the basics haven't changed.....

The Decomb filter is designed to automatically decide how to handle video. It'll deinterlace interlaced frames, detelecine the telecined frames and leave progressive frames untouched.

When you select a constant frame rate output, you'll find two lines in the log file when the decomb filter is enabled. The way Handbrake's decomb filter works for NTSC (to the best of my knowledge) is it always operates in variable frame rate mode, so it'll detelecine and/or de-interlace as required, and then drop any frames it considers to be duplicates. If you set a constant frame rate output, Handbrake may then have to drop more frames, or replace the dropped frames with duplicates again to output the requested constant frame rate.

The two lines that show what's happening in the log file look like the ones below (although it's for a PAL DVD). If you select a variable frame rate output, the second line doesn't exist in the log file. If memory serves, frames are fully de-interlaced with Yadif if a substantial amount of combing is detected, otherwise blend de-interlacing is used.

[03:37:12] decomb: deinterlaced 1092 | blended 11 | unfiltered 396 | total 1499
[03:37:12] render: 1499 frames output, 0 dropped and 0 duped for CFR/PFR

For PAL DVDs, the decomb filter default setting appears (according to my quick test above) to operate in constant frame rate mode, even when you select "variable/same as source" for the output.

I tried the "bob" setting for the decomb filter, and the following was the result for the same video ("VFR/same as source" outout):

[03:53:21] decomb: deinterlaced 2184 | blended 22 | unfiltered 396 | total 2602

From that it seems the interlaced frames were de-interlaced to 50fps (which looks smoother than de-interlacing to 25fps) and the progressive frames were left at 25fps. If you forces a constant 50fps output, the progressive frames would be duplicated.

Unlike the decomb filter, the de-interlace filter de-interlaces every frame, and in bob mode it'll de-interlace every frame to 50fps. The progressive frames will be duplicated. The downside is, it's better not to deinterlace progressive frames as it'll cause some blurring or maybe artefacts, but on the other hand the decomb filter isn't perfect at detecting which frames need de-interlacing and can leave small amounts of residual combing behind.

The above is for a PAL DVD. For NTSC where the decomb filter might need to apply detelecine, things could wok a little differently. I haven't tried test encodes using an NTSC DVD yet.

[ministry88]

It worked! The moire is gone! Thank you so much!

So I guess my revised workflow will be:

-For fully progressive DVDs and DVD-Rs of commercial content (films, TV shows) without moire artifacting present: Set Handbrake preset to "HQ 480p 30 Surround" with detelecine ON and deinterlace ON with "decomb" setting ON and framerate set to a constant 23.976.
-For fully progressive content with moire present: detelecine ON and Deinterlace OFF set to a constant 23.976 frame rate.
-For interlaced DVDs/DVD-Rs: Double the frame-rate to constant 59.94 fps (50 fps for PAL) and use "decomb."

Of course the workflow is subject to twiddling of the aforementioned settings based on my spot checks of the DVDs, but at least this workflow has cut a path through the jungle. I have a few additional questions:

-Is there a diagnostic tool to detect if a DVD is progressive, interlaced, 3:2 pulldown, or a mix? Or should I just spot-check the frames in slow-mo looking for combing using VLC?
-My new encode ended up in the m4v format, which I think is a proprietary Apple format. Do you know why it came out in m4v format and not in mp4? Strangely, when I chose to convert the file to "original quality" in Plex, it came out as mp4! Weird.
-Do I need to double the frame rate for fully progressive or mixed interlaced/progressive video?

[hello_hello]

I just played around with an NTSC source (I made a short clip with hard-coded 3:2 pulldown myself for testing) and I appear to have posted some incorrect information originally. I thought the decomb filter would apply detelecine to telecined sections as required, but it appears it simply de-interlaces them, so the result is 29.976fps where every fifth frame is repeated.
For DVDs with soft pulldown (not included) it appears Handbrake ignores the pulldown flags, so it can handle progressive 23.976fps content, the decomb filter will de-interlace the interlaced video sections to 29.970 and output a variable frame rate if required, but it seems there's no way to handle a mixture of soft and hard-coded pulldown and interlaced NTSC in an ideal way. You have to put up with repeated frames for any hard-telecine sections, or enable the detelecine filter and risk interlace video sections having IVTC applied.

I guess that explains many of the old Handbrake hybrid NTSC encodes I've seen that have been a mixture of "film" with every fifth frame repeated and 29.970fps progressive.

It worked! The moire is gone! Thank you so much!

So I guess my revised workflow will be:

-For fully progressive DVDs and DVD-Rs of commercial content (films, TV shows) without moire artifacting present: Set Handbrake preset to "HQ 480p 30 Surround" with detelecine ON and deinterlace ON with "decomb" setting ON and framerate set to a constant 23.976.
-For fully progressive content with moire present: detelecine ON and Deinterlace OFF set to a constant 23.976 frame rate.

The older version of Handbrake I have doesn't let you enable decomb and de-interlacing at the same time.
The main risk to setting a constant 23.976fps or enabling the detelecine filter is for DVDs with 29.970 interlaced sections they'll be converted to 23.976 by dropping frames. I still think the Handbrake default of decomb enabled, same as source VFR, is the best compromise, and if the source is progressive and the decomb filter causes problems, just disable it.

-For interlaced DVDs/DVD-Rs: Double the frame-rate to constant 59.94 fps (50 fps for PAL) and use "decomb."

Decomb set to "bob" if you want double frame rate de-interlacing.

-Is there a diagnostic tool to detect if a DVD is progressive, interlaced, 3:2 pulldown, or a mix? Or should I just spot-check the frames in slow-mo looking for combing using VLC?

Most of us open the source via Avisynth and step through the frames, but you can probably do the same with VLC if you make sure it's de-interlacing is disabled. If you see a 3:2 pattern of clean and combed frames where there's movement, it hard- telecine, if every frame is combed, it's interlaced. Hopefully you won't see much 3:2 combing because most DVDs would have the 23.976fps sections encoded as progressive with pulldown flags, and Handbrake/VLC should ignore them when decoding and all you'll see is the progressive video.

My new encode ended up in the m4v format, which I think is a proprietary Apple format. Do you know why it came out in m4v format and not in mp4? Strangely, when I chose to convert the file to "original quality" in Plex, it came out as mp4! Weird.
-Do I need to double the frame rate for fully progressive or mixed interlaced/progressive video?

m4v is basically mp4 with the addition of DRM support. I don't know when Handbrake chooses m4v exactly, but try manually changing the extension to mp4 before you start encoding.
If you don't include any AC3 audio, I think Handbrake will default to MP4 instead. Maybe it's related to PlayStation support, as mentioned here. https://en.wikipedia.org/wiki/M4V
MP4 didn't originally support AC3, although I think it's pretty common now.

[poisondeathray]

m4v is Apple's modified MP4 container. It's their way of saying "MP4 with AC3 audio", because the original MP4 spec did not include AC3 support . m4v is more compatible with Apple devices when you have AC3 audio

But just to confuse everybody, "m4v" can also denote the extension for Adobe's elementary AVC/h.264 video stream

[manono]

-Is there a diagnostic tool to detect if a DVD is progressive, interlaced, 3:2 pulldown, or a mix? Or should I just spot-check the frames in slow-mo looking for combing using

You can run the Preview using DGIndex, after opening the VOB(s). However, both hard telecine and true interlace will show as the same thing, but if it's a mix of hard and soft telecine (from a film DVD), it'll show that. If worse comes to worst, you'd have to examine the frames.

Why anyone in NTSC land uses Handbrake at all is beyond me. These kinds of problems don't exist when using AviSynth.