VideoHelp Forum
+ Reply to Thread
Results 1 to 10 of 10
Thread
  1. Hi all,

    I am capturing and upscaling some old movies I'm adding to my Plex library. I use Topaz AI to upscale to 4K, which takes about a day using all the horsepower in my NVIDIA RTX card and CPU, but it creates fantastic results. These are mostly 1990s and earlier films I have that were never released on Blu Ray. Otherwise my library is all raw rips of Blu Rays and UHD Blu Rays I bought over the years. Topaz AI in this case outputs a raw ProRes file which can be upwards of 1 TB. I then use ffmpeg and NVENC to near losslessly compress it down to the normal ~60-70 GB range.

    This has worked flawlessly on the old films I have used this process on. I recently however came a cross a film that I cannot get to play back without serious lag. It is a film from the 1990s only ever released in what looks like ~720p, and that's the best quality I can buy. I run it through Topaz AI to upscale to 4K, which takes about a day. Then I take that file and run it through ffmpeg NVENC which gets it down to like 70 GB in about an hour. However this file will not play without serious lag despite having a bitrate around 60 MB/s, the same as all my 4K files and UHD Blu Ray rips.

    When I seek through the file frame by frame, all the frames are there, so there isn't a problem with the stream. After seeking frame by frame a bit, it will then play smoothly for a few seconds, before stuttering and lagging. This happens with VLC and MPC HD on the PC, and with anything I try to play it back with. The only thing that has been able to play it back smoothly is my NVIDIA Shield TV through Plex.

    The one thing that does differentiate this file is the source is 59.940 fps. All films are basically 23.976 fps, with a small portion in 24p, and with a very small portion of TV content at 29.976 fps. This confused me, so I took the source video and used ffmpeg to turn the 59.940 fps down to 23.976. The weirdest thing was that in some scenes of the film, there are duplicate frames even at that fps, which means the source is some kind of variable fps, or drops to sub 23.976 at certain scenes in the film. I also tried 24 fps, 29.976, and 30 fps. These had even more duplicate frames so the closest to the proper frame rate for the content appears to be 23.976.

    However this output had serious judder problems. After doing a comprehensive comparison, the smoothest frame rate to watch was in fact the 59.940 version. I tried using Topaz AI's sophisticated interpolation feature to remove duplicate frames and create new frames at a real 23.976 FPS, but this resulted in some jarring judder since the motion was not 100% smooth since it was using AI to interpolate.

    So that leaves me using the above Reference quality HDMI capture>AI upscale>ffmpeg NVENC compression>MKV w/ proper metadata process which has worked perfectly with every other film, in this case at 59.940 fps with the duplicate frames.

    However this final output, despite a normal bitrate and normal settings from what I can tell, will not play smoothly on almost any device, which I cannot figure out. The judder is also a bit more notable, perhaps because more detail makes it more pronounced than in the original, but it would be fine if it just wouldn't lag.

    To add a couple more details, the original capture pre-upscale from the source does not have the lag issue. It's a TV movie from HBO from the 1990s that it seems like they've published/encoded in a weird way. The source signal gives off 1920x1080 @ bt709 @ 59.940 fps.

    Here is the Media Info of the source file:

    Code:
    General
    Unique ID                                : 266443290003576894874236379002616100045 (0xC8731C6F139ABB32FF244F77B97334CD)
    Complete name                            :movie to upscale.mkv
    Format                                   : Matroska
    Format version                           : Version 4
    File size                                : 22.1 GiB
    Duration                                 : 1 h 41 min
    Overall bit rate                         : 31.2 Mb/s
    Encoded date                             : UTC 2023-08-28 13:26:54
    Writing application                      : mkvmerge v67.0.0 ('Under Stars') 64-bit
    Writing library                          : libebml v1.4.2 + libmatroska v1.6.4 / Lavf60.3.100
    
    Video
    ID                                       : 1
    Format                                   : HEVC
    Format/Info                              : High Efficiency Video Coding
    Format profile                           : Main 10@L5.1@High
    Codec ID                                 : V_MPEGH/ISO/HEVC
    Duration                                 : 1 h 41 min
    Bit rate                                 : 30.6 Mb/s
    Width                                    : 1 920 pixels
    Height                                   : 1 080 pixels
    Display aspect ratio                     : 16:9
    Frame rate mode                          : Constant
    Frame rate                               : 59.940 FPS
    Color space                              : YUV
    Chroma subsampling                       : 4:2:0
    Bit depth                                : 10 bits
    Bits/(Pixel*Frame)                       : 0.246
    Stream size                              : 21.7 GiB (98%)
    Writing library                          : Lavc60.3.100 hevc_nvenc
    Default                                  : No
    Forced                                   : No
    Color range                              : Limited
    Color primaries                          : BT.709
    Transfer characteristics                 : BT.709
    Matrix coefficients                      : BT.709
    
    Audio
    ID                                       : 2
    Format                                   : AC-3
    Format/Info                              : Audio Coding 3
    Commercial name                          : Dolby Digital
    Codec ID                                 : A_AC3
    Duration                                 : 1 h 41 min
    Bit rate mode                            : Constant
    Bit rate                                 : 640 kb/s
    Channel(s)                               : 6 channels
    Channel layout                           : L R C LFE Ls Rs
    Sampling rate                            : 48.0 kHz
    Frame rate                               : 31.250 FPS (1536 SPF)
    Bit depth                                : 32 bits
    Compression mode                         : Lossy
    Delay relative to video                  : 23 ms
    Stream size                              : 464 MiB (2%)
    Writing library                          : Lavc60.3.100 ac3
    Service kind                             : Complete Main
    Default                                  : No
    Forced                                   : No
    Here is the Media Info of the upscaled file post Topaz AI:

    Code:
    General
    Complete name                            : upscaled movie_prob3.mov
    Format                                   : MPEG-4
    Format profile                           : QuickTime
    Codec ID                                 : qt   0000.02 (qt  )
    File size                                : 719 GiB
    Duration                                 : 1 h 41 min
    Overall bit rate mode                    : Variable
    Overall bit rate                         : 1 015 Mb/s
    Encoded date                             : UTC 2023-08-28 13:26:54
    Tagged date                              : UTC 2023-08-28 13:26:54
    major_brand                              : qt  
    minor_version                            : 512
    compatible_brands                        : qt  
    creation_time                            : 2023-08-28T13:26:54.000000Z
    videoai                                  : Enhanced using prob-3 with recover details at 20.3686, dehalo at 17.616100000000003, reduce noise at 4.05132, sharpen at 50, revert compression at 59, and anti-alias/deblur at 26.649099999999997. Changed resolution to 3840x2160
    encoder                                  : Lavf60.4.100
    
    Video
    ID                                       : 1
    Format                                   : ProRes
    Format version                           : Version 0
    Format profile                           : 422 LT
    Codec ID                                 : apcs
    Duration                                 : 1 h 41 min
    Bit rate mode                            : Variable
    Bit rate                                 : 1 015 Mb/s
    Width                                    : 3 840 pixels
    Height                                   : 2 160 pixels
    Display aspect ratio                     : 16:9
    Frame rate mode                          : Constant
    Frame rate                               : 59.940 FPS
    Color space                              : YUV
    Chroma subsampling                       : 4:2:2
    Scan type                                : Progressive
    Bits/(Pixel*Frame)                       : 2.041
    Stream size                              : 719 GiB (100%)
    Writing library                          : Apple
    Encoded date                             : UTC 2023-08-28 13:26:54
    Tagged date                              : UTC 2023-08-28 13:26:54
    Color primaries                          : BT.709
    Transfer characteristics                 : BT.709
    Matrix coefficients                      : BT.709
    
    Audio
    ID                                       : 2
    Format                                   : AC-3
    Format/Info                              : Audio Coding 3
    Codec ID                                 : ac-3
    Duration                                 : 1 h 41 min
    Source duration                          : 1 h 41 min
    Bit rate mode                            : Constant
    Bit rate                                 : 640 kb/s
    Channel(s)                               : 6 channels
    Channel layout                           : L C R Ls Rs LFE
    Sampling rate                            : 48.0 kHz
    Compression mode                         : Lossy
    Stream size                              : 464 MiB (0%)
    Source stream size                       : 464 MiB (0%)
    Default                                  : Yes
    Alternate group                          : 1
    Encoded date                             : UTC 2023-08-28 13:26:54
    Tagged date                              : UTC 2023-08-28 13:26:54
    Here is the Media Info of the post ffmpeg NVENC compression of the upscaled file:

    Code:
    General
    Unique ID                                : 29989895220746140207177789974341364267 (0x168FD7FE42D9E8CCEA9F47BA8C9A2A2B)
    Complete name                            : Finished movie.mkv
    Format                                   : Matroska
    Format version                           : Version 4
    File size                                : 51.7 GiB
    Duration                                 : 1 h 41 min
    Overall bit rate                         : 73.0 Mb/s
    Encoded date                             : UTC 2023-09-07 18:34:43
    Writing application                      : mkvmerge v67.0.0 ('Under Stars') 64-bit
    Writing library                          : libebml v1.4.2 + libmatroska v1.6.4 / Lavf60.3.100
    VIDEOAI                                  : Enhanced using prob-3 with recover details at 20.3686, dehalo at 17.616100000000003, reduce noise at 4.05132, sharpen at 50, revert compression at 59, and anti-alias/deblur at 26.649099999999997. Changed resolution to 3840x2160
    
    Video
    ID                                       : 1
    Format                                   : HEVC
    Format/Info                              : High Efficiency Video Coding
    Format profile                           : Format Range@L5.1@High
    Codec ID                                 : V_MPEGH/ISO/HEVC
    Duration                                 : 1 h 41 min
    Bit rate                                 : 72.4 Mb/s
    Width                                    : 3 840 pixels
    Height                                   : 2 160 pixels
    Display aspect ratio                     : 16:9
    Frame rate mode                          : Constant
    Frame rate                               : 59.940 FPS
    Color space                              : YUV
    Chroma subsampling                       : 4:4:4
    Bit depth                                : 10 bits
    Bits/(Pixel*Frame)                       : 0.146
    Stream size                              : 51.3 GiB (99%)
    Writing library                          : Lavc60.3.100 hevc_nvenc
    Language                                 : English
    Default                                  : Yes
    Forced                                   : No
    Color range                              : Limited
    Color primaries                          : BT.709
    Transfer characteristics                 : BT.709
    Matrix coefficients                      : BT.709
    VENDOR_ID                                : FFMP
    
    Audio
    ID                                       : 2
    Format                                   : AC-3
    Format/Info                              : Audio Coding 3
    Commercial name                          : Dolby Digital
    Codec ID                                 : A_AC3
    Duration                                 : 1 h 41 min
    Bit rate mode                            : Constant
    Bit rate                                 : 640 kb/s
    Channel(s)                               : 6 channels
    Channel layout                           : L R C LFE Ls Rs
    Sampling rate                            : 48.0 kHz
    Frame rate                               : 31.250 FPS (1536 SPF)
    Bit depth                                : 32 bits
    Compression mode                         : Lossy
    Delay relative to video                  : 18 ms
    Stream size                              : 464 MiB (1%)
    Language                                 : English
    Service kind                             : Complete Main
    Default                                  : Yes
    Forced                                   : No
    VENDOR_ID                                : [0][0][0][0]
    
    Text
    ID                                       : 3
    Format                                   : UTF-8
    Codec ID                                 : S_TEXT/UTF8
    Codec ID/Info                            : UTF-8 Plain Text
    Duration                                 : 1 h 38 min
    Bit rate                                 : 77 b/s
    Frame rate                               : 0.324 FPS
    Count of elements                        : 1923
    Stream size                              : 56.3 KiB (0%)
    Language                                 : English
    Default                                  : Yes
    Forced                                   : No
    
    Menu
    00:00:00.000                             : Chapter 01
    Here is the ffmpeg command that was used to compress:

    Code:
    ffmpeg -i input.mov -c:a copy -c:v hevc_nvenc -b:v 60M -bufsize:v 120M -maxrate 70M -minrate 50M -g 4 -r 59.940 -s 3840x2160 output.mkv
    Here is the ffmpeg command that was used to capture the original movie:

    Code:
    ffmpeg -f dshow -c:v v210 -video_pin_name 1 -audio_pin_name 1 -video_size 1920x1080 -rtbufsize 2147480000 -framerate 60000/1001 -channels 6 -sample_rate 48k -i video="Decklink Video Capture":audio="Decklink Audio Capture" -filter_complex "[0:0]scale=w=iw:h=ih:interl=false:eval=init:out_v_chr_pos=-128:out_h_chr_pos=-128:flags=bilinear+accurate_rnd+print_info,format=p010le[vout];[0:1]channelmap=FL-FL|FR-FR|LFE-FC|FC-LFE|BL-BL|BR-BR:5.1[aout]" -map "[vout]" -map "[aout]" -color_range 1 -chroma_sample_location left -color_trc bt709 -color_primaries bt709 -colorspace bt709 -strict -2 -c:a ac3 -b:a 6144k -c:v hevc_nvenc -profile:v main10 -preset:v p7 -b:v 80M -bufsize:v 120M -maxrate 90M -minrate 70M -g 4 -t 02:10:00 movie.mkv
    Basically my question is why would this file be lagging like this (a behavior of a movie with a bitrate too high for the playback hardware) when the bitrate and parameters seem like they should be able to play fine, and seeking frame by frame reveals all the frames are there and allows a buffer of a few seconds of smooth playback, and one particular device (NVIDIA Shield TV) is able to play the file back without issue? My process/hardware works on every other movie of this sort, so it doesn't seem like a hardware limitation since I can play UHD Blu Ray rips just fine. It seems like the only difference is that this is 59.940 fps vs. the usual 23.976, but I can't imagine that alone causes such lag.

    Any help would be appreciated. Thanks!
    Last edited by Boxcart; 8th Sep 2023 at 13:42.
    Quote Quote  
  2. Many players/devices will not play 10 bit 4:4:4 HEVC at 59.94 fps.
    Last edited by jagabo; 9th Sep 2023 at 09:03.
    Quote Quote  
  3. You should go from 59.94 (60000/1001) fps to 29.97 (30000/1001) fps, not to 23.976 (24000/1001) fps.
    Last edited by ProWo; 9th Sep 2023 at 10:31.
    Quote Quote  
  4. Originally Posted by jagabo View Post
    Many players/devices will not play 10 bit 4:4:4 HEVC at 59.94 fps.
    I figured it was the frame rate since that's the only thing that's different with this particular film. I used 10 bit and 4:4:4 since I use a modified command to capture 4K HDR so it's easier to just change the resolution and color and keep the rest of the command the same. What command could I use to change this file to 8 bit 4:2:2 with minimal disruption? Right now I have it in an MKV with subtitles, chapters, proper metadata, etc. I'd like to drop it into ffmpeg and have it fix that but I've never done 10 bit to 8 bit with ffmpeg before. Thanks.
    Quote Quote  
  5. Originally Posted by ProWo View Post
    You should go from 59.94 (60000/1001) fps to 29.97 (30000/1001) fps, not to 23.976 (24000/1001) fps.
    I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.
    Quote Quote  
  6. Member
    Join Date
    Jun 2022
    Location
    Dublin
    Search Comp PM
    The -g 4 appears to me to be a bit extreme. Would say -g 60 work better?
    Going to 444 from 422 is probably only padding out, probably unnecessary.
    Quote Quote  
  7. Originally Posted by JN- View Post
    The -g 4 appears to me to be a bit extreme. Would say -g 60 work better?
    Going to 444 from 422 is probably only padding out, probably unnecessary.
    I could do that but would it solve this problem? I use -g 4 to have more frequent key frames to make trimming with MKVToolnix easier. If 4:2:2 is unnecessary what ffmpeg command should I use to go from 10 bit to 8 bit with minimal loss/disruption? Thanks
    Last edited by Boxcart; 9th Sep 2023 at 13:22.
    Quote Quote  
  8. Originally Posted by Boxcart View Post
    I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.
    More duplicates as the original with 59.94 fps?
    I dont think so.
    The source was ntsc telecined from 23.976 to 29.97 (introducing duplicates) and then framerate doubled to 59.94.
    At 29.97 should play without lag.
    If so, you can inverse telecine the 29.97 to 23.976.
    Quote Quote  
  9. Originally Posted by ProWo View Post
    Originally Posted by Boxcart View Post
    I tried 29.97 but that results in more duplicate frames since it seems like HBO's source is 23.976, but with certain scenes even giving duplicate frames at that frame rate. In other words the source from HBO was mastered very poorly in the 1990s.
    More duplicates as the original with 59.94 fps?
    I dont think so.
    The source was ntsc telecined from 23.976 to 29.97 (introducing duplicates) and then framrate duobled to 59.94.
    At 29.97 should play without lag.
    If so, you can inverse telecine the 29.97 to 23.976.
    Not more duplicates at 29.97 than 59.94 (more at 29.97 than 23.976), but worse judder at 29.97 than at 59.94. There are panning and driving scenes that look okay at 59.94, but when reduced to 29.97 are unbearable. I'm currently using ffmpeg to go from 10 bit to 8 bit (command I came up with and am using is below) and I'll see if that fixes the lag issue when it's done. I agree though I think HBO in the 1990s took the 23.976 (or 24) source and telecined up to 29.97 for TV, and then in the digital age doubled it to 59.94. In fact a lot of their stuff from that time frame is like this. I haven't run into it with any other material I have, only HBO stuff. In fact I have some HBO documentaries recorded from my TiVo raw from the cable MPEG-2 stream that give the signal of 29.97, but when you actually seek frame by frame are in this same weird gray area of frame rate with duplicate frames where going down to 23.976 or 24 just introduces judder. TV's now with interpolation tend to smooth it out partly but it's very imperfect.

    10 bit to 8 bit command:

    Code:
    ffmpeg -i input.mkv -c:a copy -c:v hevc_nvenc -b:v 70M -bufsize:v 120M -maxrate 80M -minrate 60M -pix_fmt yuv420p -r 59.940 -s 3840x2160 output.mkv
    Last edited by Boxcart; 9th Sep 2023 at 14:38.
    Quote Quote  
  10. Update: Re-encoding to 8 bit fixed the lag problem
    Quote Quote  



Similar Threads

Visit our sponsor! Try DVDFab and backup Blu-rays!