Mysterious video won't play without lag

[Boxcart]

Hi all,

I am capturing and upscaling some old movies I'm adding to my Plex library. I use Topaz AI to upscale to 4K, which takes about a day using all the horsepower in my NVIDIA RTX card and CPU, but it creates fantastic results. These are mostly 1990s and earlier films I have that were never released on Blu Ray. Otherwise my library is all raw rips of Blu Rays and UHD Blu Rays I bought over the years. Topaz AI in this case outputs a raw ProRes file which can be upwards of 1 TB. I then use ffmpeg and NVENC to near losslessly compress it down to the normal ~60-70 GB range.

This has worked flawlessly on the old films I have used this process on. I recently however came a cross a film that I cannot get to play back without serious lag. It is a film from the 1990s only ever released in what looks like ~720p, and that's the best quality I can buy. I run it through Topaz AI to upscale to 4K, which takes about a day. Then I take that file and run it through ffmpeg NVENC which gets it down to like 70 GB in about an hour. However this file will not play without serious lag despite having a bitrate around 60 MB/s, the same as all my 4K files and UHD Blu Ray rips.

When I seek through the file frame by frame, all the frames are there, so there isn't a problem with the stream. After seeking frame by frame a bit, it will then play smoothly for a few seconds, before stuttering and lagging. This happens with VLC and MPC HD on the PC, and with anything I try to play it back with. The only thing that has been able to play it back smoothly is my NVIDIA Shield TV through Plex.

The one thing that does differentiate this file is the source is 59.940 fps. All films are basically 23.976 fps, with a small portion in 24p, and with a very small portion of TV content at 29.976 fps. This confused me, so I took the source video and used ffmpeg to turn the 59.940 fps down to 23.976. The weirdest thing was that in some scenes of the film, there are duplicate frames even at that fps, which means the source is some kind of variable fps, or drops to sub 23.976 at certain scenes in the film. I also tried 24 fps, 29.976, and 30 fps. These had even more duplicate frames so the closest to the proper frame rate for the content appears to be 23.976.

However this output had serious judder problems. After doing a comprehensive comparison, the smoothest frame rate to watch was in fact the 59.940 version. I tried using Topaz AI's sophisticated interpolation feature to remove duplicate frames and create new frames at a real 23.976 FPS, but this resulted in some jarring judder since the motion was not 100% smooth since it was using AI to interpolate.

So that leaves me using the above Reference quality HDMI capture>AI upscale>ffmpeg NVENC compression>MKV w/ proper metadata process which has worked perfectly with every other film, in this case at 59.940 fps with the duplicate frames.

However this final output, despite a normal bitrate and normal settings from what I can tell, will not play smoothly on almost any device, which I cannot figure out. The judder is also a bit more notable, perhaps because more detail makes it more pronounced than in the original, but it would be fine if it just wouldn't lag.

To add a couple more details, the original capture pre-upscale from the source does not have the lag issue. It's a TV movie from HBO from the 1990s that it seems like they've published/encoded in a weird way. The source signal gives off 1920x1080 @ bt709 @ 59.940 fps.

Here is the Media Info of the source file:

General
Unique ID                                : 266443290003576894874236379002616100045 (0xC8731C6F139ABB32FF244F77B97334CD)
Complete name                            :movie to upscale.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 22.1 GiB
Duration                                 : 1 h 41 min
Overall bit rate                         : 31.2 Mb/s
Encoded date                             : UTC 2023-08-28 13:26:54
Writing application                      : mkvmerge v67.0.0 ('Under Stars') 64-bit
Writing library                          : libebml v1.4.2 + libmatroska v1.6.4 / Lavf60.3.100

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 h 41 min
Bit rate                                 : 30.6 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.246
Stream size                              : 21.7 GiB (98%)
Writing library                          : Lavc60.3.100 hevc_nvenc
Default                                  : No
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : A_AC3
Duration                                 : 1 h 41 min
Bit rate mode                            : Constant
Bit rate                                 : 640 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Bit depth                                : 32 bits
Compression mode                         : Lossy
Delay relative to video                  : 23 ms
Stream size                              : 464 MiB (2%)
Writing library                          : Lavc60.3.100 ac3
Service kind                             : Complete Main
Default                                  : No
Forced                                   : No

Here is the Media Info of the upscaled file post Topaz AI:

General
Complete name                            : upscaled movie_prob3.mov
Format                                   : MPEG-4
Format profile                           : QuickTime
Codec ID                                 : qt   0000.02 (qt  )
File size                                : 719 GiB
Duration                                 : 1 h 41 min
Overall bit rate mode                    : Variable
Overall bit rate                         : 1 015 Mb/s
Encoded date                             : UTC 2023-08-28 13:26:54
Tagged date                              : UTC 2023-08-28 13:26:54
major_brand                              : qt  
minor_version                            : 512
compatible_brands                        : qt  
creation_time                            : 2023-08-28T13:26:54.000000Z
videoai                                  : Enhanced using prob-3 with recover details at 20.3686, dehalo at 17.616100000000003, reduce noise at 4.05132, sharpen at 50, revert compression at 59, and anti-alias/deblur at 26.649099999999997. Changed resolution to 3840x2160
encoder                                  : Lavf60.4.100

Video
ID                                       : 1
Format                                   : ProRes
Format version                           : Version 0
Format profile                           : 422 LT
Codec ID                                 : apcs
Duration                                 : 1 h 41 min
Bit rate mode                            : Variable
Bit rate                                 : 1 015 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:2
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 2.041
Stream size                              : 719 GiB (100%)
Writing library                          : Apple
Encoded date                             : UTC 2023-08-28 13:26:54
Tagged date                              : UTC 2023-08-28 13:26:54
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Codec ID                                 : ac-3
Duration                                 : 1 h 41 min
Source duration                          : 1 h 41 min
Bit rate mode                            : Constant
Bit rate                                 : 640 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L C R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Stream size                              : 464 MiB (0%)
Source stream size                       : 464 MiB (0%)
Default                                  : Yes
Alternate group                          : 1
Encoded date                             : UTC 2023-08-28 13:26:54
Tagged date                              : UTC 2023-08-28 13:26:54

Here is the Media Info of the post ffmpeg NVENC compression of the upscaled file:

General
Unique ID                                : 29989895220746140207177789974341364267 (0x168FD7FE42D9E8CCEA9F47BA8C9A2A2B)
Complete name                            : Finished movie.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 51.7 GiB
Duration                                 : 1 h 41 min
Overall bit rate                         : 73.0 Mb/s
Encoded date                             : UTC 2023-09-07 18:34:43
Writing application                      : mkvmerge v67.0.0 ('Under Stars') 64-bit
Writing library                          : libebml v1.4.2 + libmatroska v1.6.4 / Lavf60.3.100
VIDEOAI                                  : Enhanced using prob-3 with recover details at 20.3686, dehalo at 17.616100000000003, reduce noise at 4.05132, sharpen at 50, revert compression at 59, and anti-alias/deblur at 26.649099999999997. Changed resolution to 3840x2160

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Format Range@L5.1@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 1 h 41 min
Bit rate                                 : 72.4 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 FPS
Color space                              : YUV
Chroma subsampling                       : 4:4:4
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.146
Stream size                              : 51.3 GiB (99%)
Writing library                          : Lavc60.3.100 hevc_nvenc
Language                                 : English
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
VENDOR_ID                                : FFMP

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : A_AC3
Duration                                 : 1 h 41 min
Bit rate mode                            : Constant
Bit rate                                 : 640 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Bit depth                                : 32 bits
Compression mode                         : Lossy
Delay relative to video                  : 18 ms
Stream size                              : 464 MiB (1%)
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No
VENDOR_ID                                : [0][0][0][0]

Text
ID                                       : 3
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 1 h 38 min
Bit rate                                 : 77 b/s
Frame rate                               : 0.324 FPS
Count of elements                        : 1923
Stream size                              : 56.3 KiB (0%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Menu
00:00:00.000                             : Chapter 01

Here is the ffmpeg command that was used to compress:

ffmpeg -i input.mov -c:a copy -c:v hevc_nvenc -b:v 60M -bufsize:v 120M -maxrate 70M -minrate 50M -g 4 -r 59.940 -s 3840x2160 output.mkv

Here is the ffmpeg command that was used to capture the original movie:

ffmpeg -f dshow -c:v v210 -video_pin_name 1 -audio_pin_name 1 -video_size 1920x1080 -rtbufsize 2147480000 -framerate 60000/1001 -channels 6 -sample_rate 48k -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc bt709 -color_primaries bt709 -colorspace bt709 -strict -2 -c:a ac3 -b:a 6144k -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -bufsize:v 120M -maxrate 90M -minrate 70M -g 4 -t 02:10:00 movie.mkv

Basically my question is why would this file be lagging like this (a behavior of a movie with a bitrate too high for the playback hardware) when the bitrate and parameters seem like they should be able to play fine, and seeking frame by frame reveals all the frames are there and allows a buffer of a few seconds of smooth playback, and one particular device (NVIDIA Shield TV) is able to play the file back without issue? My process/hardware works on every other movie of this sort, so it doesn't seem like a hardware limitation since I can play UHD Blu Ray rips just fine. It seems like the only difference is that this is 59.940 fps vs. the usual 23.976, but I can't imagine that alone causes such lag.

Any help would be appreciated. Thanks!

[jagabo]

Many players/devices will not play 10 bit 4:4:4 HEVC at 59.94 fps.

[ProWo]

You should go from 59.94 (60000/1001) fps to 29.97 (30000/1001) fps, not to 23.976 (24000/1001) fps.

[Boxcart]

Many players/devices will not play 10 bit 4:4:4 HEVC at 59.94 fps.

I figured it was the frame rate since that's the only thing that's different with this particular film. I used 10 bit and 4:4:4 since I use a modified command to capture 4K HDR so it's easier to just change the resolution and color and keep the rest of the command the same. What command could I use to change this file to 8 bit 4:2:2 with minimal disruption? Right now I have it in an MKV with subtitles, chapters, proper metadata, etc. I'd like to drop it into ffmpeg and have it fix that but I've never done 10 bit to 8 bit with ffmpeg before. Thanks.

[Boxcart]

You should go from 59.94 (60000/1001) fps to 29.97 (30000/1001) fps, not to 23.976 (24000/1001) fps.

I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.

[JN-]

The -g 4 appears to me to be a bit extreme. Would say -g 60 work better?
Going to 444 from 422 is probably only padding out, probably unnecessary.

[Boxcart]

The -g 4 appears to me to be a bit extreme. Would say -g 60 work better?
Going to 444 from 422 is probably only padding out, probably unnecessary.

I could do that but would it solve this problem? I use -g 4 to have more frequent key frames to make trimming with MKVToolnix easier. If 4:2:2 is unnecessary what ffmpeg command should I use to go from 10 bit to 8 bit with minimal loss/disruption? Thanks

[ProWo]

I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.

More duplicates as the original with 59.94 fps?
I dont think so.
The source was ntsc telecined from 23.976 to 29.97 (introducing duplicates) and then framerate doubled to 59.94.
At 29.97 should play without lag.
If so, you can inverse telecine the 29.97 to 23.976.

[Boxcart]

I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.

More duplicates as the original with 59.94 fps?
I dont think so.
The source was ntsc telecined from 23.976 to 29.97 (introducing duplicates) and then framrate duobled to 59.94.
At 29.97 should play without lag.
If so, you can inverse telecine the 29.97 to 23.976.

Not more duplicates at 29.97 than 59.94 (more at 29.97 than 23.976), but worse judder at 29.97 than at 59.94. There are panning and driving scenes that look okay at 59.94, but when reduced to 29.97 are unbearable. I'm currently using ffmpeg to go from 10 bit to 8 bit (command I came up with and am using is below) and I'll see if that fixes the lag issue when it's done. I agree though I think HBO in the 1990s took the 23.976 (or 24) source and telecined up to 29.97 for TV, and then in the digital age doubled it to 59.94. In fact a lot of their stuff from that time frame is like this. I haven't run into it with any other material I have, only HBO stuff. In fact I have some HBO documentaries recorded from my TiVo raw from the cable MPEG-2 stream that give the signal of 29.97, but when you actually seek frame by frame are in this same weird gray area of frame rate with duplicate frames where going down to 23.976 or 24 just introduces judder. TV's now with interpolation tend to smooth it out partly but it's very imperfect.

10 bit to 8 bit command:

ffmpeg -i input.mkv -c:a copy -c:v hevc_nvenc -b:v 70M -bufsize:v 120M -maxrate 80M -minrate 60M -pix_fmt yuv420p -r 59.940 -s 3840x2160 output.mkv

[Boxcart]

Update: Re-encoding to 8 bit fixed the lag problem