Getting rid of duplicate frames in a 30p video sourced from 24p

[alexpigment]

Hey guys. I've got several videos captured from my Hauppauge HD PVR 2 and I'm trying to a) eliminate judder from duplicate frames, and b) author to 1080p24 Blu-ray. The shows in question are natively 24fps (or 23.98fps), but since the HD PVR 2 captures 1080p60 (the framerate of the HDMI source) as 1080p30, there are duplicate frames.

I tried converting with Handbrake to both 23.98fps and 24fps, which I believe enforces a DCT-Decimate filter. The problem is that the judder will come and go in intervals. Some areas are correctly decimated, so the result is smoother than the 30fps files; other areas are incorrectly decimated, and the result is worse than the 30fps files.

I have several paid tools at my disposal: VideoReDo, Premiere Pro, as well as the whole Adobe creative suite. I'm thinking that neither of those will help, and I may be forced to go down an AVISynth route (which I'm very unfamiliar with), but I wanted to know if it's even worth trying based on my results from Handbrake. Is it possible that the frame decimation inherent in the HD PVR's 1080p60 > 1080p30 conversion makes the problem unavoidable? Any help would be appreciated.

[hello_hello]

It's hard to know if it'll be successful without seeing a sample. Here's how you'd do it using MeGUI (Avisynth based).

Use MeGUI's File/Open menu to open the source file. MeGUI will offer to index it. Add the indexing job to the queue, switch to the Queue tab and run it. When it's done, MeGUI's Script Creator will open with a preview. Apply any cropping or resizing as required, then switch to the Filters tab. Under the de-interlacing section select "M-In-5 decimation required" as the source type. Make sure the "M" value is set to "1". Check the "De-Interlacing" checkbox and switch to the script tab. You should see something like this included:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
TDecimate(cycleR=1)

That'll remove 1 frame in 5, which in theory should be what you need. You can click on the "Preview AVS Script" button and it'll be applied to the preview. If your PC can decode fast enough you should be able to play the preview to see if it looks smooth. If not, you can step through frames one at a time.

If that's not smooth, you can try this, but you'll need to manually change the second line above so you have this:

LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\TIVTC.dll")
TDecimate(mode=7, rate=23.976)

Once you've changed it, click on the "Preview AVS Script" button again.
If neither of the above work, you might need to post a sample.

Once you've saved the script, if you switch to MeGUI's Input tab you should see the script loaded into the Video section and either a script or extracted audio in the Audio section. Select "x264 scratchpad" in the video encoder settings dropdown box, click on Config, and when the encoder configuration opens, load the defaults. Select "Bluray" in the Target Playback Device dropdown list and then adjust any other x264 settings from there (leave it set for CRF encoding for the moment by selecting "targeting quality"). The audio encoder is configured in a similar manner, but for AAC you need to either manually download NeroAAC, FDK-AAC, or the required files for QAAC to work, and enable them in MeGUI's options. NeroAAC is easiest as it doesn't require additional files. Just put NeroAacEnc.exe in the "MeGUI\tools\eac3to" folder. Details for downloading the other AAC encoders can be found here.

Once the encoders are set up click on AutoEncode down the bottom of MeGUI's main window. When AutoEncode opens you can select a file size if need be (MeGUI will switch to 2 pass encoding if you do) or you can use the CRF value you selected when configuring the encoder (by not specifying a file size). You can also add additional audio etc, but when you click on Queue MeGUI will add several jobs to the queue. Switch to the Queue tab and click "Start".

That's the basics of using MeGUI. If you need further help, someone will be able to offer it.

[alexpigment]

Wow. I didn't expect that level of help! I've tested one file and it appears to look good, so hopefully the same will hold true for the rest. Thank you again for this immense help. No need to worry about the AAC btw (which isn't Blu-ray compliant anyway). I've already demuxed my AC3 audio, and Adobe Encore will remux it with the H.264 stream for me during the authoring process. Thanks!

[hello_hello]

No problem.
MeGUI can be a bit "where do I start?" if you've never used it before, and I had a bit of time to write instructions. Plus you're now on your way to learning Avisynth.

If you've already demuxed the audio (MeGUI can also do it for you for certain file types) you can use the Queue button in the video section to encode just the video if you want to (rather than use AutoEncode), and do the remuxing later.

For Bluray compliance, it pays to check MeGUI's log file after you start encoding. If Bluray is selected as the target playback device in the x264 encoder configuration and your encode won't result in a 100% Bluray compatible output (due to resolution or frame rate etc) you should find warnings in the log file.

Oh.... and you can confirm the output video specifications by adding
Info()
to the end of a script and clicking the "Preview AVS Script" button in the script creator. Just make sure to remove Info() before saving the script for encoding.

[alexpigment]

I've had to do quite a lot of work in FFMPEG and x264 writing custom profiles, so Blu-ray compatibility hasn't been too much of a problem for me. I usually refer to this guide if I don't have the parameters readily available:

http://www.x264bluray.com/home/1080i-p

The scripting and filters are really where I'm still green, so I appreciate your specific instructions for MeGUI. As for the audio, Adobe Encore is fairly particular about having demuxed elementary streams as input, at least in my experience. I've never had any luck feeding it MKV, TS, or M2TS files - it always just assumes it needs to transcode. Not a huge problem, but it makes my project folders a bit more messy than I'd like them to be

[hello_hello]

I've had to do quite a lot of work in FFMPEG and x264 writing custom profiles, so Blu-ray compatibility hasn't been too much of a problem for me. I usually refer to this guide if I don't have the parameters readily available:

http://www.x264bluray.com/home/1080i-p

Just so you know how MeGUI works, if you select Bluray as the target playback device in MeGUI's x264 encoder configuration, MeGUI should prevent you from changing any of x264's settings to something that isn't Bluray compatible, including the advanced x264 options.
If you leave the GOP (keyint) settings at their defaults in the encoder configuration, MeGUI normally adjusts the GOP size to a maximum of 10x the frame rate. You won't see it in the x264 configuration command line because it changes according to frame rate, but for the default x264 settings at 23.976fps MeGUI will set keyint=240 and keyint_min=23 unless you over-ride it, and with Bluray selected as the target playback device, it'll set keyint=24 & keyint_min=1.

It'll ensure --colorprim "bt709" --transfer "bt709" & --colormatrix "bt709" when encoding HD for Bluray, but it's not shown in the command line either because for standard definition they change to bt470bg or smpte170m.

I didn't know people still encoded for Bluray compatibility. I thought everything played MKVs these days.