FFMPEG transcoded MPEG-TS files are lagging to start video

[dogmydog]

Hello everyone,

Thanks to the help of this forum and ProWo, I have been able to generate TS files from my MP4 files using ffmpeg.
for %i in (*.mp4) do ffmpeg -i "%i" -c:v h264_nvenc -preset slow -r 29.97 -profile:v main -level:v 4.0 -cbr true -b:v 6M -minrate 6M -maxrate 6M -bufsize 8M -muxrate 8M -c:a copy "%~ni.ts"

Problem is that when I start playing and skipping forward the TS files (on VLC, MPC-HC, etc.), the audio starts right away, but the video takes 3-4 seconds to start.
Any ideias on why this might be occurring? Should I add the "-movflags +faststart " to the encode?

Thank you for the attention,

[Selur]

Should I add the "-movflags +faststart " to the encode?

Unless you change your output container to mov, mp4, ismv that's not doing to do anything (assuming ffmpeg just ignores it and doesn't abort).

Any ideias on why this might be occurring?

without details,...
input might be vfr,..
time codes might be broken in some way,...

--

First thing I would try is adding the '-r 29.97' additionally before the input to make sure input is decoded as cfr. see: https://www.ffmpeg.org/ffmpeg-all.html#Video-Options

Cu Selur

[dogmydog]

Should I add the "-movflags +faststart " to the encode?

Unless you change your output container to mov, mp4, ismv that's not doing to do anything (assuming ffmpeg just ignores it and doesn't abort).

Any ideias on why this might be occurring?

without details,...
input might be vfr,..
time codes might be broken in some way,...

--

First thing I would try is adding the '-r 29.97' additionally before the input to make sure input is decoded as cfr. see: https://www.ffmpeg.org/ffmpeg-all.html#Video-Options

Cu Selur

Wow thanks for the rapid feedback, @Cu Selur!
I have tried that, but the results are very similar. I have recorded the MPC window using OBS to show the differences.

EXAMPLE 1: file generated using FFMPEG
https://youtu.be/UmCKOlXahUo

When I encode the TS file on a program called Episode 6, the problem is not there.
EXAMPLE 2: file generated using EPISODE 6 PRO
https://youtu.be/tF5uIHkGnHQ

Thanks again for the help!

[DB83]

I am curious as to why you need to go from one container format .mp4 to another .ts


Post a mediainfo (text mode) report of the original .mp4 and the transcoded .ts

[dogmydog]

Client's platform only runs TS, not Mp4.
Here is the original media info
General
Complete name : B:\CurtaCT Drive\VOD\TS\2000993.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42 (mp42/mp41)
File size : 2.76 GiB
Duration : 24 min 54 s
Overall bit rate mode : Variable
Overall bit rate : 15.8 Mb/s
Encoded date : UTC 2021-04-09 14:50:49
Tagged date : UTC 2021-04-09 14:51:38
TIM : 00:00:00:00
TSC : 30000
TSZ : 1001

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4.1
Format settings : CABAC / 3 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 3 frames
Format settings, GOP : M=1, N=30
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 24 min 54 s
Source duration : 24 min 54 s
Bit rate : 15.5 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.250
Stream size : 2.70 GiB (98%)
Source stream size : 2.70 GiB (98%)
Language : English
Encoded date : UTC 2021-04-09 14:50:49
Tagged date : UTC 2021-04-09 14:50:49
Codec configuration box : avcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 24 min 54 s
Source duration : 24 min 54 s
Bit rate mode : Variable
Bit rate : 317 kb/s
Maximum bit rate : 416 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 56.6 MiB (2%)
Source stream size : 56.6 MiB (2%)
Language : English
Encoded date : UTC 2021-04-09 14:50:49
Tagged date : UTC 2021-04-09 14:50:49

Here is the transcoded by ffmpeg:
General
ID : 1 (0x1)
Complete name : B:\CurtaCT Drive\VOD\TS\2000993.ts
Format : MPEG-TS
File size : 1.39 GiB
Duration : 24 min 54 s
Overall bit rate mode : Constant
Overall bit rate : 8 000 kb/s

Video
ID : 256 (0x100)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4
Format settings : CABAC / 3 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 3 frames
Codec ID : 27
Duration : 24 min 54 s
Bit rate mode : Constant
Nominal bit rate : 6 000 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.097

Audio
ID : 257 (0x101)
Menu ID : 1 (0x1)
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Format version : Version 4
Muxing mode : ADTS
Codec ID : 15-2
Duration : 24 min 54 s
Bit rate mode : Variable
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Language : English

Menu
ID : 4096 (0x1000)
Menu ID : 1 (0x1)
Duration : 24 min 54 s
List : 256 (0x100) (AVC) / 257 (0x101) (AAC, English)
Language : / English
Service name : Service01
Service provider : FFmpeg
Service type : digital television

And here is the Episode 6 specs:
General
ID : 0 (0x0)
Complete name : E:\VOD\netnow\1012094.ts
Format : MPEG-TS
File size : 3.72 GiB
Duration : 53 min 12 s
Overall bit rate mode : Constant
Overall bit rate : 10 000 kb/s

Video
ID : 481 (0x1E1)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Main@L4
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Codec ID : 27
Duration : 53 min 11 s
Bit rate mode : Constant
Nominal bit rate : 6 000 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Scan type, store method : Interleaved fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.116
Color range : Limited

Audio
ID : 482 (0x1E2)
Menu ID : 1 (0x1)
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Format version : Version 2
Muxing mode : ADTS
Codec ID : 15-2
Duration : 53 min 11 s
Bit rate mode : Variable
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Delay relative to video : -40 ms

[DB83]

I am no expert but my guess is that there is a compatability issue between the high-bitrate AV1 codec in the first video to your re-encoded/transcoded AVC lower-bitrate.




The second sample is originally AVC so that was my 'marker'

[dogmydog]

I am no expert but my guess is that there is a compatability issue between the high-bitrate AV1 codec in the first video to your re-encoded/transcoded AVC lower-bitrate.

The second sample is originally AVC so that was my 'marker'

Thank you for this input, but this doesn't explain why I can convert the same video in FFMPEG and have this lag on my players and using the same video on Episode 6, there is no lag with audio starting then video

[DB83]

I guess your client has also indicated certain restrictions in bitrate etc. So a simple remux from .mp4 to .ts is off the table


Not sure whether having constant bitrate for the video but variable bitrate for the audio (as per the original) is also a factor.


You could always upload a short sample of the original as an attachment (not to youtube who will encode it again). But the new encode per the youtube sample appears to be in sync in normal playback.

[dogmydog]

I guess your client has also indicated certain restrictions in bitrate etc. So a simple remux from .mp4 to .ts is off the table


Not sure whether having constant bitrate for the video but variable bitrate for the audio (as per the original) is also a factor.


You could always upload a short sample of the original as an attachment (not to youtube who will encode it again). But the new encode per the youtube sample appears to be in sync in normal playback.

Dear @DB83 thanks for the feeback!

Please note that this thread is not about a sync problem between audio and video, but this:
The TS file generated by FFMPEG, when you skip ahead in time playing in VLC or MPC, that audio starts aprox. 3 seconds before the video (without losing sync).
The TS file generated by Episode6, when you skip ahead in time playing in VLC or MPC, that audio and video play at the same time, just like in the original .mp4 file (without losing sync).

I have uploades 3 sample files to this wetransfer link:
https://we.tl/t-0RjrywldUG
original.mp4 is the original file used to generate the FFMPEG TS
episode.ts is the file generated by Episode 6
ffmpeg.ts is the file generated by FFMPEG

If you open all in MPC or VLC, and try to skip ahead in time, youll notice when you jump forward on the TS generated by FFMPEG (ffmpeg.ts), the video freezes for about 2-3 seconds while the audio starts instantaneouly. When you jump forward the original.mp4 file or episode6.ts, the video and audio start almost at the same time.

Hopefully that wasn't too complicated lol
I REALLY appreciate your help and support given to this matter. Have a great day!

[Selur]

1st observation:
According to details infos from mediainfo (--full) the ffmpeg version has:
a. a delay of 1 s 400 ms for the video
and
b. a 701ms delay for the audio
that seems strange.

2nd observation:
Max gop size of the ffmpeg.ts is 250
Max gop size of the episode6.ts is 180
Max gop size of the original.mp4 is 180
-> I would probably use an even shorter gop size (150 or 100; '-g 100')
Reencoding the video stream of the ffmpeg.ts with a gop size of 100 fixes jumping issues here.

ffmpeg -i "c:\Users\Selur\Desktop\wetransfer-1f7c98\original.mp4" -c:v h264_nvenc -preset slow -r 29.97 -profile:v main -level:v 4.0 -g 100 -cbr true -b:v 6M -minrate 6M -maxrate 6M -bufsize 8M -muxrate 8M -c:a copy "e:\Output\test.ts"

(only thing I changed was add '-g 100' and adjust the paths.

Cu Selur

[DB83]

I am no expert but my guess is that there is a compatability issue between the high-bitrate AV1 codec in the first video to your re-encoded/transcoded AVC lower-bitrate.

The second sample is originally AVC so that was my 'marker'

Thank you for this input, but this doesn't explain why I can convert the same video in FFMPEG and have this lag on my players and using the same video on Episode 6, there is no lag with audio starting then video

I thought I had attempted to explain that but I now see you only posted the report 'result' of 'Episode 6' and not the source info. Even so I do note a slight variation in that report inasmuch that there are 2 reference frames rather than 3 in the other video.


Our friend Selur might now have solved your issue. However, if the encode was meant for broadcast, normal playback would prevail. Transport files (.ts) do have more 'baggage' than normal streams due to their 'packet' nature (a viewer not having to watch from the start) and .mp4 do typically have long gops. I have many such files and when I skip forward there is typically visual disturbance.

[dogmydog]

However, if the encode was meant for broadcast, normal playback would prevail. Transport files (.ts) do have more 'baggage' than normal streams due to their 'packet' nature (a viewer not having to watch from the start) and .mp4 do typically have long gops. I have many such files and when I skip forward there is typically visual disturbance.

Dear DB83, thanks for this explanation.
I agree with you, I understand that .ts files are more "laggy" than MP4, and I only use that format because that is what client's platform demands.

It is a VOD platform. When it loads the titles I encoded with Episode 6, the video and audio start right away.
When it loads files encoded with FFMPEG, the audio starts right away, but the same lag I experience on VLC or MPC happens when I'm skipping forward and the video only starts about 2-3 seconds after the audio.

1st observation:
According to details infos from mediainfo (--full) the ffmpeg version has:
a. a delay of 1 s 400 ms for the video
and
b. a 701ms delay for the audio
that seems strange.

2nd observation:
Max gop size of the ffmpeg.ts is 250
Max gop size of the episode6.ts is 180
Max gop size of the original.mp4 is 180
-> I would probably use an even shorter gop size (150 or 100; '-g 100')
Reencoding the video stream of the ffmpeg.ts with a gop size of 100 fixes jumping issues here.

ffmpeg -i "c:\Users\Selur\Desktop\wetransfer-1f7c98\original.mp4" -c:v h264_nvenc -preset slow -r 29.97 -profile:v main -level:v 4.0 -g 100 -cbr true -b:v 6M -minrate 6M -maxrate 6M -bufsize 8M -muxrate 8M -c:a copy "e:\Output\test.ts"

(only thing I changed was add '-g 100' and adjust the paths.

Cu Selur

Dear @Selur, thank you for your response!
I have reencoded the file and I could see a slight improvement on the MPC playback.

But for some weird reason the file encoded by Episode is still faster to let the video "catch up" with the audio when I skip forward.
Maybe it's because it is interlaced?

Here is my standard configuration for encoding on Episode, maybe it can give you clues on why it its faster to load:
Container MPEG-TS (ts)
Transport Rate 10,000 kbit/s

Video
H264 (Main Concept)
Constant Bitrate (CBR)
Bitrate: 6,000 kbit/s
Natural and Forced Keyframes
Keyframe Interval: 100 Frames
Adaptive B-frames (X)
Number of B-frames: 3 frames
Number of Reference Frames: 2 frames

Encoder Profile: High
Context-Adaptive Binary-Arithmetic Coding (CABAC)
Color Space: 4:2:0
Display Aspect Ratio: From Source/Resize Filter
Two Pass Encoding (X)
Deblocking Filter (X)

Number of Slices: 4 Slices
Initial Buffer Fullness: 100%
IDR Frames: Every
Force headers for every GOP (X)
Signal fixed framerate (X)
Pulldown: None
Level Signalling: Level 4.0
Audio
AAC Quicktime
Bit Rate Based
Bit Rate: 160kbit/s
Setting Type: Standard Settings

I really appreciate the help that has been given to me so far, this community is incredible!

[Selur]

Maybe it's because it is interlaced?

Doesn't really seem to be interlaced,... When looking at the episode6.ts without applying any deinterlacing I see no combing.
Just seems to be saved using MBAFF without anything actually saved as interlaced.

Jumping might simply be faster if a clip contains more key frames. I only looked at the max gop size not the average, minimal gop size or anything like that.
-> If you want faster jumping use smaller max gop sizes.

[DB83]

TBH I did not look at your latest samples when I replied earlier today simply because I accepted that the issue was 'solved'.


I had hoped to actually see the samples from your original medianfo report(s) and what was uploaded to youtube. Yet I now see that these are different. For me it is somewhat difficult to suggest a solution when one deals with different sources.


And more so the new samples have no spoken dialogue thus impossible to establish potential audio-sync issues.


Now I do not have a clue how this is achieved but a .ts file typically has several different versions and the one that is most compatable to the users connection plays. That is a 1080p 6mbps might be to 'fast' whereas a 2mbps 480p will play (and even skip) fine.


But I would at the very least still prefer to see the original video pre uploaded to yt to get a better understanding of the issue.

[davexnet]

Could this be related to the usage of h264_nvenc ? For example, When I run a similar command using libx264, the .ts plays fine in mpc-hc

ffmpeg -i "C:\Users\davex\Desktop\original.mp4" -c:v libx264 -preset veryfast -r 23.976 -cbr true -b:v 6M -minrate 6M
-maxrate 6M -bufsize -8M -muxrate 8M -c:a copy test2354r.ts

[Selur]

@davexnet: yes and no.
x264 by default uses closed-gops while nvenc by default uses open-gops which is the main issue here. (since iirc. older vlc version can't properly handle open gop)
Additionally lower gops sizes also mean lower seek and sync times.

[hello_hello]

Is Fast Seek disabled for MPC-HC under Options/Tweaks? Actually it doesn't matter. I just checked and it appears appears not to work for TS files.

Normally when Fast Seek is enabled, MPC-HC jumps to the nearest keyframe, and there's only three of them in your ffmpeg sample. When it's disabled you can navigate to any frame but the entire gop needs to be decoded before video playback can start. So if you disable Fast Seek and try the attached version of your sample, remuxed as MKV, you'll find there's a similar delay, but the audio doesn't play until the video starts.

What seems to be happening for TS files, is MPC-HC always behaves as though Fast Seek is enabled to some extent, but only for the video. When you navigate, it appears to decode the keyframe after the point you navigated to, starts playing the audio straight away, while displaying the decoded keyframe until the audio catches up.

Anyway, try the remuxed version. With Fast Seek enabled you'll find there's only three places in the video you can navigate to. When it's disabed and you seek, MPC-HC displays the last frame while the group of frames it needs are being decoded, then it begins playback from the place you navigated to, but it doesn't play any audio in between, so there's still a delay, but it's both video and audio.

For x264 the default GOP size it 250. That's the maximum size (10 seconds at 25fps). The minimum GOP is 25 by default (1 second at 25fps). x264 is pretty good at putting keyframes on the first frame of a scene change and for a lengthy video you normally don't notice when navigating. It just jumps to the nearest scene change. As Selur suggested, reducing the GOP size is probably the only solution, especially as it appears for h264_nvenc the GOP size is a fixed, rather than being a maximum. Bluray uses a maximum GOP of 1 second (a GOP size of 30 for your source), so even though there'll be lots more keyframes and the file size will probably increase a bit, at least navigation should be fast.

Edit1: I see Selur mentioned h264_nvenc uses open GOP by default. So does Bluray, but it'd probably be a good thing as it'd help negate the GOP size decrease. Can you disable open GOP for h264_nvenc though?

Edfit2: I had a look and there's no B-frames in ffmpeg.ts. There's three I-frames and the rest are P-frames, so it can't be open GOP.

By the way, when I remuxed your ts file as an MKV, it went from 23.0 MB to 11.4 MB. Does the menu item ffmpeg added to the TS file bloat it out that much, or is there something else I'm not seeing?

[Selur]

By the way, when I remuxed your ts file as an MKV, it went from 23.0 MB to 11.4 MB. Does the menu item ffmpeg added to the TS file bloat it out that much, or is there something else I'm not seeing?

"-cbr true -muxrate 8M" will create bitrate stuffing

Edit1: I see Selur mentioned h264_nvenc uses open GOP by default. So does Bluray, but it'd probably be a good thing as it'd help negate the GOP size decrease. Can you disable open GOP for h264_nvenc though?

should be disabled by settings a max gop size. '-g ...'

Edfit2: I had a look and there's no B-frames in ffmpeg.ts. There's three I-frames and the rest are P-frames, so it can't be open GOP.

then the problem is just the gop size

[dogmydog]

Dear friends, it is amazing how much knowledge you have put into this matter, thanks for this.
I was ashamed on my lack of info, so I checked the blog bellow and now I feel a little less stupid
https://aws.amazon.com/pt/blogs/media/part-1-back-to-basics-gops-explained/

Since I live in a NTSC land, maybe I should change my gop to 90 or even 30?
I know this will decrease quality but maybe it will help the problem?

How about B frames? Do I need to specify like adding this:
-x264-params keyint=90:bframes=9

Or just
-g 90
would suffice?

Also, I prefer enconding on nvenc because it is SO much faster on my Geforce 1660.
But it -libc264 fixes the problem for good, maybe i should stick with it.

Thanks again for the amazing support!

[dogmydog]

Anyway, try the remuxed version. With Fast Seek enabled you'll find there's only three places in the video you can navigate to. When it's disabed and you seek, MPC-HC displays the last frame while the group of frames it needs are being decoded, then it begins playback from the place you navigated to, but it doesn't play any audio in between, so there's still a delay, but it's both video and audio.

Dear @hello_hello, I have tried the file and yes, there is only 3 points where i can click when skipping forward. Super cool!

By the way, when I remuxed your ts file as an MKV, it went from 23.0 MB to 11.4 MB. Does the menu item ffmpeg added to the TS file bloat it out that much, or is there something else I'm not seeing?

The VOD platform requires CBR, therefore I have to fix the bitrate.
Thanks for the conversion and help!