Mux 2 bluray with different fps: change audio or video speed (for quality)?

[precipizio]

Hello, everytime I need to mux different bluray with different fps, should I modify the speed of audio or the fps of the video? the goal is to preserve the quality as much as I can, and I don't know what is better to do and why. I know:
1) changing the audio speed can alter the pitch and it can't be done without re-encoding the file (for video is possible to avoid re-encoding)
2) changing the video fps should be avoided because for big changes (23.9 to 25) can completely alter the product and in animation is a bad practice because anime/cartoon animation is highly correlated to the speed between a frame and another

I don't know if my motivations are good, what is the best option in these situations? And more important: why?

[ProWo]

My 5 cent:
Change the audio, never the video.
You can change the audio (and pitch too) with clever Ffmpeg-GUI.

[precipizio]

ok, but why "never the video"? I don't need a dogma

[Cornucopia]

Video's elemental unit of time ("timebase") is the frame, which has a resolution of 1/25sec, 1/30sec, etc.
Audio's elemental unit of time is the sample, which has a resolution of 1/44100sec, 1/48000sec - aka MUCH FINER.

So when interpolating, there is more to track with audio. IOW the error is less. It is also more "fluid" (where you can shift things to a finer degree).
Plus, tools for very high quality SRC, time-stretch, pitch shift, etc. have been been around longer and are more mature. Meaning less noticeable error.

Scott

[precipizio]

Video's elemental unit of time ("timebase") is the frame, which has a resolution of 1/25sec, 1/30sec, etc.
Audio's elemental unit of time is the sample, which has a resolution of 1/44100sec, 1/48000sec - aka MUCH FINER.

So when interpolating, there is more to track with audio. IOW the error is less. It is also more "fluid" (where you can shift things to a finer degree).
Plus, tools for very high quality SRC, time-stretch, pitch shift, etc. have been been around longer and are more mature. Meaning less noticeable error.

Scott

Interesting, this is the most complete answer I got about this topic.

- if I want to rip a bluray, for example a ghibli movie that has a better video in the JAP BD than in the BD of my native language (they have different fps). In terms of quality, in your opinion is it worth to use a better video quality (JAP) in exchange for an audio stretch (my language BD), or should I give up the JAP video and use only my language BD for both audio and video? (Consider that the "better video quality" is just 3-5% more bitrate. I know this topic is subjective, but I'd like to know your opinion anyway: what should I do in these situations when I have to choose between a bit better video with audio stretching and a bit worse video without audio stretching?)

- I use archlinux OS, could you suggest some tool that can do high quality audio stretching, pitch correction etc..?
- If ffmpeg is a good solution, which filters/commands do you suggest?
- Any guide about how to do it in a "trustable" way (in terms of preserving quality) you want suggest it can be very useful for me.

Thanks for your patience.

[jagabo]

Change the audio, never the video.

Except you might be able to change the frame rate of the video losslessly (i.e. without re-encoding).

[precipizio]

Change the audio, never the video.

Except you might be able to change the frame rate of the video losslessly (i.e. without re-encoding).

Anyway it's better to change the audio speed (re-encoding it) instead of change the video fps (for example 23.9 -> 25) losslessly without re-encoding. Do you agree? Why?

[jagabo]

Losslessly changing the frame rate will prevent any quality loss of the video or audio. Any recompression of video or audio with any lossy codec will result in some loss of quality.

The issue then becomes how your playback devices deal with different frame rates. For example most 50 Hz displays will convert lower frame rates to 50p by simple duplication. A 25 fps source will be converted to 50 fps by displaying every source frame twice. But that won't work with a 24 fps source as the result would be 48 fps, not 50 fps. So most source frames will be displayed for the duration of 2 output frames, but every 12th will be displayed for 3 output frames. So every 12 input frames becomes 25 output frames, 24 input frames becomes 50 output frames. That creates 2 little jerks every second. It will be obvious in smooth panning shots or other smooth motions. In such a case you may decide that a small loss of audio quality is preferable to getting slightly jerky video.

[precipizio]

Losslessly changing the frame rate will prevent any quality loss of the video or audio. Any recompression of video or audio with any lossy codec will result in some loss of quality.

The issue then becomes how your playback devices deal with different frame rates. For example most 50 Hz displays will convert lower frame rates to 50p by simple duplication. A 25 fps source will be converted to 50 fps by displaying every source frame twice. But that won't work with a 24 fps source as the result would be 48 fps, not 50 fps. So most source frames will be displayed for the duration of 2 output frames, but every 12th will be displayed for 3 output frames. So every 12 input frames becomes 25 output frames, 24 input frames becomes 50 output frames. That creates 2 little jerks every second. It will be obvious in smooth panning shots or other smooth motions. In such a case you may decide that a small loss of audio quality is preferable to getting slightly jerky video.

Thank you very much, this is the best answer I got on this topic!

[precipizio]

Losslessly changing the frame rate will prevent any quality loss of the video or audio. Any recompression of video or audio with any lossy codec will result in some loss of quality.

The issue then becomes how your playback devices deal with different frame rates. For example most 50 Hz displays will convert lower frame rates to 50p by simple duplication. A 25 fps source will be converted to 50 fps by displaying every source frame twice. But that won't work with a 24 fps source as the result would be 48 fps, not 50 fps. So most source frames will be displayed for the duration of 2 output frames, but every 12th will be displayed for 3 output frames. So every 12 input frames becomes 25 output frames, 24 input frames becomes 50 output frames. That creates 2 little jerks every second. It will be obvious in smooth panning shots or other smooth motions. In such a case you may decide that a small loss of audio quality is preferable to getting slightly jerky video.

I have a couple of doubts:

1) all conversions you quoted (50 Hz displays will convert lower frame rates to 50p by simple duplication [...] most source frames will be displayed for the duration of 2 output frames, but every 12th will be displayed for 3 output frames [.. etc..]) are referred to any video, not only videos with losslessly changed fps. So, for example, every 24fps video will be displayed by showing input frames for the duration of 2 output frames, but every 12th will be displayed for 3 output frames. Is the general idea correct?

2) "In such a case you may decide that a small loss of audio quality is preferable to getting slightly jerky video." These cases are specific rare cases or almost every video with fps losslessly changed has such a jerky video? (I'm talking about mainstream live action and anime/cartoon movies)

My goal is to find a general logic to mux video/audio from different sources with different fps. So I'd like to know if generally I have to prioritize speed up/down of audio or changing fps of video. And how to practically recognize situations in which it's better to do the opposite.

Thanks for your time and patience.

[Cornucopia]

There is not overarching general logic, there are workflow case best practices based on the background (I like to call this provenance) of the clips, and the presumed intermediate workflow that has already been performed on it. You first have to determine what the true background of the clip is, and then change your correctional workflow based on that. There are rules but there are many exceptions to the rules.

Scott

[precipizio]

There is not overarching general logic, there are workflow case best practices based on the background (I like to call this provenance) of the clips, and the presumed intermediate workflow that has already been performed on it. You first have to determine what the true background of the clip is, and then change your correctional workflow based on that. There are rules but there are many exceptions to the rules.

Scott

Ok, anyway I don't know why famous encoders/releasers argue that losslessly change the frame timestamps introduce video jerks: when does it happen? I'd like to know generally how frames timestamps are "moved" (in the timeline) in order to know some if problems are introduced.

thanks for support and patience.

[davexnet]

It happens the way jagabo described in post #8 above

[precipizio]

Losslessly changing the frame rate will prevent any quality loss of the video or audio. Any recompression of video or audio with any lossy codec will result in some loss of quality.

The issue then becomes how your playback devices deal with different frame rates. For example most 50 Hz displays will convert lower frame rates to 50p by simple duplication. A 25 fps source will be converted to 50 fps by displaying every source frame twice. But that won't work with a 24 fps source as the result would be 48 fps, not 50 fps. So most source frames will be displayed for the duration of 2 output frames, but every 12th will be displayed for 3 output frames. So every 12 input frames becomes 25 output frames, 24 input frames becomes 50 output frames. That creates 2 little jerks every second. It will be obvious in smooth panning shots or other smooth motions. In such a case you may decide that a small loss of audio quality is preferable to getting slightly jerky video.

I'm not sure to fully understand, could you tell me if it's correct?

- If I losslessly change fps by changing frames timestamps without re-encoding the video -> I can get little jerks every second (or periodically) depending on the monitor
- If I don't change video fps in any way, so I re-encode only audio in order to change its speed -> I can't get any jerks in video, I could get a little quality loss in the audio track, depending on the encoding

[jagabo]

No matter what frame rate the source is you will jerks when the monitor's refresh rate isn't the same (or integer multiple thereof). So if you watch a 25 fps video on a 50 Hz monitor you will no get get any jerks. Or if you watch a 24 Hz video on a 72 Hz monitor you will not get any jerks.

When you change the timecodes of a video you are simply telling the player to play the video at a different rate. So changing a 24p video to 25 tells the player to display each frame for 1/25 seconds (40 ms) instead of 1/24 seconds (41.666... ms). That changes the running time of the video but does not effect the quality. And changing the timecodes is much much faster than re-encoding the video.

[precipizio]

Ok, thank you. So generally it seems the best way to mux sources at different FPS is to change timestamps of frames, so no encode is done and the full quality is preserved. if I understand correctly. Maybe the only remaining doubt can be if changing timestamp is a good for anime/cartoon. I suppose it's good since there is only a proportional decrease/increase of time speed, so each frame will be distant from its neighbors in a proportionally identical way as it was before the timestamp change.

if everything is correct, I don't know why all encoders/muxers online prefer to encode audio instead of changing frames timestamps.

[manono]

In terms of quality, in your opinion is it worth to use a better video quality (JAP) in exchange for an audio stretch (my language BD), or should I give up the JAP video and use only my language BD for both audio and video?[/B]

Neither. Watch the original and better Japanese video with the original Japanese audio with subtitles in your own language.

[precipizio]

In terms of quality, in your opinion is it worth to use a better video quality (JAP) in exchange for an audio stretch (my language BD), or should I give up the JAP video and use only my language BD for both audio and video?[/B]

Neither. Watch the original and better Japanese video with the original Japanese audio with subtitles in your own language.

(.-.)