VFR to CFR conversion before editing

[droner]

I am trying to convert my drone VFR video (no audio) to CFR before editing.
When i use handbrake there seems to be duplicated frames added to the video which cause stutter.
The stutter is worst when the image is moving in the footage.
As i don't have any audio in the drone footage i don't care about any audio sync problems.
I have read that you can rewrap the MP4 video and only change the presentation time stamp.
I am new to video editing and don't understand some of the technical term used.
I have tried mp4fpsmod command but it won't read my mp4 file.
Or is there a FFMPEG command line that i can use to convert without adding duplicated frames.
I would appreciate if anyone could help.

[poisondeathray]

I am trying to convert my drone VFR video (no audio) to CFR before editing.
When i use handbrake there seems to be duplicated frames added to the video which cause stutter.
The stutter is worst when the image is moving in the footage.
As i don't have any audio in the drone footage i don't care about any audio sync problems.
I have read that you can rewrap the MP4 video and only change the presentation time stamp.
I am new to video editing and don't understand some of the technical term used.
I have tried mp4fpsmod command but it won't read my mp4 file.
Or is there a FFMPEG command line that i can use to convert without adding duplicated frames.
I would appreciate if anyone could help.

That's what converting to CFR usually means. Periods of lower FPS in your VFR footage get duplicate frames added to make up CFR. You're missing frames compared to if your drone shot CFR in the first place .

If you just assumed that frame rate, those sections would be "sped up" in speed during those lower fps shot sections, and appear to slow down during the "normal" sections

There is a small chance that handbrake messed up, but it's unlikely . If you post a sample with motion someone can check it out to verify

[droner]

Maybe i wasn't using handbrake properly.
My VFR goes from 28.944 to 30.777 fps.
So when i fix the frame rate to 30fps duplicated frames are added when my video is at 28.944 fps and frames are dropped when the video is at 30.77fps
Here is a link to the stuttering video.
https://youtu.be/8mZXhvzSkZQ

[poisondeathray]

Maybe i wasn't using handbrake properly.
My VFR goes from 28.944 to 30.777 fps.
So when i fix the frame rate to 30fps duplicated frames are added when my video is at 28.944 fps and frames are dropped when the video is at 30.77fps
Here is a link to the stuttering video.
https://youtu.be/8mZXhvzSkZQ

It would be more helpful if you posted the original file, because you're complicating the issues witheyoutube re-encoding and compression , and possibly mishandling the file

Frames get dropped and duplicated - that is what you expect should happen for a standard CFR conversion.

If you assume assume a constant frame rate, or change the PTS - it will appear it speed up and slow down and stutter in sections, because of the timestamps in your original file (equivalent to dropped frames in your source) , which appears just as bad as stuttering. Essentially you're missing frames, that's the bottom line.

The other option is motion interpolation (optical flow), or interpolating over the duplicates inserted with synthesized "in-between" frames . There are functions in avisynth such as filldrops and derivative functions which detect duplicate frames and attempt to fix them. This is the smoothest option, but motion interpolation doesn't always work, sometimes there are artifacts

It's a beautiful scenery shot - too bad the drone shoots VFR

[Cornucopia]

If the clips are within the ~±1.1FPS variation of 29.97 you mention, the speedup/slowdown stutter wouldn't be that noticeable except in sections with familiar, predictable or rhythmic motion. That is quite small, so you could fool it and force it if you needed to.

Scott

[poisondeathray]

Unfortunately that drone video is all rhythmic, familiar motion .

A gap or jump in motion from a dropped frame with the VFR slowtime timestamp (longer display time on that frame) is as noticable as the stutter from an inserted duplicated. If you assume a constant frame rate, there will be a jump in motion. If you're someone that notices one, you should notice the other

It's like snowing and -30 celcius where I am, and that video looks like where I would like to be right now

[smrpix]

What software are you using to edit? Some, like Premiere are very good at handling VFR footage these days and require no pre-conversion. They also allow other modes such as blending or interpolating rather than simple drop or duplicate.

[hello_hello]

MediaInfo often reports VFR for MP4 when it's really CFR. Try remuxing the original MP4 as an MKV with MKVToolNix to see if MediaInfo changes it's mind.

I know I've already mentioned MediaInfo but has droner mentioned how he determined the source was VFR and also the min/max frame rate etc.

And yes, a sample of the source would be a tad more helpful than a stutter re-encode uploaded to YouTube.

[droner]

Thanks for your comments
I am using MediaInfo to get the video information.
My original VFR file is 3.6gb 4k video so might be to big to upload on here.
And i don't know how to cut it without rendering it. Maybe if i installed LosslessCut could do it without changing the original format or rerendering.
I have had some luck with adjusting the time code to FFR 30fps with mp4fpsmod.
Then rendering it in vegas pro.
Looks like vegas pro with the standard setting was resampling the footage as the rendering speed was slower and fluctuating compared to rendering with smart resampling off.
The resampled footage seem to be not as sharp as when i turned of smart resample in vegas pro.
I read somewhere that resampling can cause blur in vegas pro and most people turn it off.
I will try to attach the 2 rendered files one with what i think is resampled and one without resampling.
Both files are 1080p 30fps
A couple of questions
Do you think the resampled footage is blurry
Is there much stutter in the footage with no resampling.
And when i use mp4fpsmod to adjust time code for later rendering in vegas pro at 29.97fps, should i set mp4fpsmod to 30fps or 30000/1001

[poisondeathray]

Yes, you can call the resampled version more "blurry", as more frames are resampled, you can see it with the tree detail. Frames are made of blends between adjacent frames

Both videos have compression artifacts and keyframe popping, it's unclear if that was from your vegas render or if had that in your original




If you set it to 30.0 in mp4fpsmod, that's what you should use in vegas for both (1) project settings and (2) render settings

Check what vegas "thinks" the framerate is in the clip bin

If things match, there should be no difference between smart resampling on or off. Things only get resampled when you have a mismatch somewhere

But if you're rendering at "29.97", that' s what you should set it to in mp4fpsmod as 30000/1001

[poisondeathray]

In the no resample version, every 1000 frames there is a jump in motion as a frame is dropped.
33sec, 66sec (1min 06s) etc....

Those drops should be eliminated by matching the framerates. 30.0 is not the same thing as 30000/1001. Vegas is dropping the frames. If you rendered it as 30.0 from vegas, (or set it to 30000/1001 in mp4fpsmod and rendered it as 29.97 in vegas ,with matching project settings), those drops should be fixed because vegas is the one dropping the frames from your mismatch


The good news is some the duplicates in the youtube version, do not appear to have corresponding drops in the mp4fpsmod vegas version with no resample. You might be able to get away with just using mp4fpsmod, and setting vegas up with matching settings. I would have expected dropped frames near the same locations as the duplicates in the YT version, but I don't see them. Something else must have gone wrong with that version

EDIT: but there is a stutter around 1:18 with an unexplained duplicate frame. If you set mp4fpsmod to 30.0, and vegas to 29.97, you'd expect drops not duplicates. Something else is going on, or it might be in your original , or some other error

[droner]

Thanks Poison death ray.
I can see that stutter too at 1:18 into the not resampled video.
The youtube footage was drone footage recorded at 4k 30fps vfr and i render in vegas pro at 1080p 29.97fps for friend to view on HD TV.

I notice the stutter is more pronounced when the drone is moving over the scene.
I assume with faster movement across the ground @30fps there is a bigger difference in each frame image, so when you get a duplicated frame the stutter is more noticeable.
After seeing the stuttering on the 4k 30fps rendered footage, I started to record in 2.7k 60fps vfr which is a lot smoother with very little stutter at all.
But i would still like to record in 4k@30fps as that the highest combined resolution and frame rate my drone can do.
When i import the mp4fpsmod @30fps file into vegas it detects it as 4k@30fps. And the project properties say 4k@30fps.
But when i select to render with MAGIX AVC 1080p @29.97 there is no option to render @30fps.
I thought 30 fps was close enough/assumed as 29.97fps
So from what you said, to get the framerates right i set mp4fpsmod @30000/1001(29.97fps) then render in vegas @29.97

With the video i attached i set mp4fpsmod to 30fps that could be why i still had a slight stutter when render in vegas @29.97fps
Could this also be why when vegas pro default setting of smart resample, tried to resample because of the two different frame rates.
Even though i didn't have mp4fpsmod set to 30000/1001 but instead at 30fps the not resampled rendered video i attached has a lot less stutter than the utube video that was rendered direct from the drome's vfr source video into vegas pro @29.97fps

In some of the utube video you can see a 4sec repetitive stutter.

Attached is the video using mp4fpsmod @30000/1001 the render in vegas pro @29.97 smart sample enabled.
Rendered fast and smoothly. I don't think vegas tried to resample or detected a timing problem.
There was a stutter in the original drone footage at the start just before the big tree but it seem to have gone.

Please check it out and point out anything you see that is wrong with it.
Glad i came across the site.
Been searching all over the internet for days trying to find a fix.
Thank you all for your help.

[jagabo]

A VFR video that varies from 28.944 to 30.777 fps is essentially CFR. You can force it to CFR by remuxing to a container that doesn't support VFR (like AVI) at whatever frame rate you want, then remux back to MP4. You will retain every frame of the source, the conversion will be very fast, motion will be perfectly smooth (assuming the chosen frame rate matches the display rate), and there will be no quality loss. With ffmpeg:

ffmpeg -r 30000.0/1001.0 -i VFR.MP4 -c:v copy -an CFR.avi
ffmpeg -fflags +genpts -i CFR.AVI -c:v copy -an CFR.mp4
del CFR.AVI

The runtime may be slightly different than the original (depending on the frame rate chosen) but it probably doesn't matter with this type of video.

[poisondeathray]

Attached is the video using mp4fpsmod @30000/1001 the render in vegas pro @29.97 smart sample enabled.
Rendered fast and smoothly. I don't think vegas tried to resample or detected a timing problem.
There was a stutter in the original drone footage at the start just before the big tree but it seem to have gone.

Those issues are gone. That glitch corresponding to 1:18 is also gone, motion looks ok...

But still issues with compression artifacts and keyframe popping . If you watch the trees, the quality goes bad and then pops into focus. It's the quality difference between I,B,P frames in long GOP. It creates a sort of cyclical flicker. It creates the impression of less smooth. If your source didn't have that, you could probably use better render settings or encoder like voukoder, libx264 to eliminate it (but something like youtube compression will create artifacts like that again). If you source did have that, not much you can do about it, unless you want to degrade/smooth over all the frames to make them more consistent, but that reduces detail

[_Al_]

It looks like B frames are blurred as oppose P or I Frames.

[_Al_]

In that zip file, there are PNG's with one GOP crops , 14 frames (I BBBP BBBP BBBP P + one I new frame) , you can see details and text what frame it is

[droner]

Attached is the video using mp4fpsmod @30000/1001 the render in vegas pro @29.97 smart sample enabled.
Rendered fast and smoothly. I don't think vegas tried to resample or detected a timing problem.
There was a stutter in the original drone footage at the start just before the big tree but it seem to have gone.

Those issues are gone. That glitch corresponding to 1:18 is also gone, motion looks ok...

But still issues with compression artifacts and keyframe popping . If you watch the trees, the quality goes bad and then pops into focus. It's the quality difference between I,B,P frames in long GOP. It creates a sort of cyclical flicker. It creates the impression of less smooth. If your source didn't have that, you could probably use better render settings or encoder like voukoder, libx264 to eliminate it (but something like youtube compression will create artifacts like that again). If you source did have that, not much you can do about it, unless you want to degrade/smooth over all the frames to make them more consistent, but that reduces detail

In the original drone footage there could be flicker from the drone propellers when the sun is at a certain angle.
At some stage i did step through frame by frame where there was the approx 4sec stutter and saw duplicated frames every 140 frames.

I'm only new to editing and don't understand a lot about frame structure in the video.
I have added the voukoder codecs to vegas pro, so i could give them a go.
When i render the last video i attached i only used default quality and upped the VBR 14000 to 24000b/sec with 1080p res
What does the number of slices mean in the MAGIX custom settings. It was set at 4 but can go to32.
What are the better codec's to use in vegas pro. There are so many to choose from.

[droner]

A VFR video that varies from 28.944 to 30.777 fps is essentially CFR. You can force it to CFR by remuxing to a container that doesn't support VFR (like AVI) at whatever frame rate you want, then remux back to MP4. You will retain every frame of the source, the conversion will be very fast, motion will be perfectly smooth (assuming the chosen frame rate matches the display rate), and there will be no quality loss. With ffmpeg:

ffmpeg -r 30000.0/1001.0 -i VFR.MP4 -c:v copy -an CFR.avi
ffmpeg -fflags +genpts -i CFR.AVI -c:v copy -an CFR.mp4
del CFR.AVI

The runtime may be slightly different than the original (depending on the frame rate chosen) but it probably doesn't matter with this type of video.

I have done a ffmpeg using the npdecimate instruction to find duplicated frames.
I did the conversion to AVI using the instruction ffmpeg -r 30000.0/1001.0 -i VFR.MP4 -c:v copy -an CFR.avi.
The file played back with stutter's VLC media player.
It was better played back on windows media play but had a section of what looked like black frames every 4 sec.

[poisondeathray]

I'm only new to editing and don't understand a lot about frame structure in the video.
I have added the voukoder codecs to vegas pro, so i could give them a go.
When i render the last video i attached i only used default quality and upped the VBR 14000 to 24000b/sec with 1080p res
What does the number of slices mean in the MAGIX custom settings. It was set at 4 but can go to32.
What are the better codec's to use in vegas pro. There are so many to choose from.

slices are how the picture is divided up into sections of macroblocks. Each section is separate. More slices has to potential to positively affect threading (and thus speed) for encoding and decoding (but also depends on the specific encoder or decoder), at the cost of encoding efficiency (quality is always lower at a given bitrate when using more slices). Some situations require a specific number of slices for compatibility, such as L4.1 AVC blu-ray always requires 4

In general, a very high bitrate will fix all lossy compression problems, unless those problems were in your source to begin with. Vegas' AVC codecs (Magix/Sony, Mainconcept) do not produce very good quality and not very efficient. You need a signifcantly higher bitrate (thus filesize) to reach a certain level of quality. They cause problems like the ones in your video at average bitrates, because the allocation of bitrate between I,B,P is not very good.

Are the problems in your source to begin with? - that's the question . You can use avidemux in "copy" mode to stream copy a section. You need to mark in/mark out a section cutting on keyframes by navigating using the "<<" ">>" buttons in the GUI. Those buttons advance to the nearest keyframe. Push the "A" and "B" buttons in the GUI to mark in , mark out. File=>save . MKV container , or MP4 container are fine as long as the video output mode says "copy" .

[droner]

In that zip file, there are PNG's with one GOP crops , 14 frames (I BBBP BBBP BBBP P + one I new frame) , you can see details and text what frame it is

I had a look at the frames
Frame 378 seemed to be the sharpest.
Sorry i don't understand the GOP crop frame stuff.
Thanks for spending the time reviewing but as a amateur video editor it's a bit hard to interpret.

[droner]

I'm only new to editing and don't understand a lot about frame structure in the video.
I have added the voukoder codecs to vegas pro, so i could give them a go.
When i render the last video i attached i only used default quality and upped the VBR 14000 to 24000b/sec with 1080p res
What does the number of slices mean in the MAGIX custom settings. It was set at 4 but can go to32.
What are the better codec's to use in vegas pro. There are so many to choose from.

slices are how the picture is divided up into sections of macroblocks. Each section is separate. More slices has to potential to positively affect threading (and thus speed) for encoding and decoding (but also depends on the specific encoder or decoder), at the cost of encoding efficiency (quality is always lower at a given bitrate when using more slices). Some situations require a specific number of slices for compatibility, such as L4.1 AVC blu-ray always requires 4

In general, a very high bitrate will fix all lossy compression problems, unless those problems were in your source to begin with. Vegas' AVC codecs (Magix/Sony, Mainconcept) do not produce very good quality and not very efficient. You need a signifcantly higher bitrate (thus filesize) to reach a certain level of quality. They cause problems like the ones in your video at average bitrates, because the allocation of bitrate between I,B,P is not very good.

Are the problems in your source to begin with? - that's the question . You can use avidemux in "copy" mode to stream copy a section. You need to mark in/mark out a section cutting on keyframes by navigating using the "<<" ">>" buttons in the GUI. Those buttons advance to the nearest keyframe. Push the "A" and "B" buttons in the GUI to mark in , mark out. File=>save . MKV container , or MP4 container are fine as long as the video output mode says "copy" .

I installed avidemux and cut a portion of the original drone VFR video.
Saved it as a MKV file.
Played it back with VLC and it's got the stutters.
See attached file.
If i was to render with voukoder codecs should i select the project default settings.
And adjust vegus pro project setting to change resolution eg1080p that i require.
Having trouble attaching file will try later

[jagabo]

I did the conversion to AVI using the instruction ffmpeg -r 30000.0/1001.0 -i VFR.MP4 -c:v copy -an CFR.avi.
The file played back with stutter's VLC media player.
It was better played back on windows media play but had a section of what looked like black frames every 4 sec.

It doesn't matter how the AVI file plays (AVC in AVI can be problematic anyway). What matters is how the following CFR MP4 file works in your editor.

[_Al_]

Thanks for spending the time reviewing but as a amateur video editor it's a bit hard to interpret.

Having those images in some folder, those png images, you just double click the first one , windows default image viewer would pop up with image, press button to get full screen (rectangle button at the bottom, middle) and then use arrows on the keyboard to move to the other images.

As pdr pointed out and brought attention to it, some frames are pretty worse quality than others, in this concrete case B frames (as if heavily blurred). Which could be Vegas encoding problem or your camera problem. But guessing it is encoding problem because of that B frame regularity.

Your encoded video has a 14 frame GOP, that means I frame followed by bunch of B and P frames , where 15th frame is again I frame that starts following GOP etc.
Those frame go like this: I BBBP BBBP BBBP P and then I frame again for next GOP etc.

So inequality is pretty big. So you have sharp I frame followed by three B unsharp B frames, then one sharp P frame etc...

I'd try that Voucoder if you get it working with Vegas. Or other built in encoder, there is two perhaps, not sure what is left there in that new Magix version. It used to be Sony avc and MainConcept avc.

[droner]

I did have a look at the images with windows photo and did see one was sharper than the other frames.
Still trying to upload the original drone VFR video to see if the blurred frames are in the source video
I have installed Voukoder for vegas pro.
So will try rendering with it.

[droner]

Didn't realize there is a 500kb limit on upload.
That why i couldn't upload.
Attached is my original 4k drone VFR video cut with Avidemux.
Maybe the blurred frames are in the original drone footage.

[jagabo]

I remuxed your MKV back into an MP4 file to restore its VFR state. Then ran the script I gave earlier. The result was a CFR mp4 file that you could probably load directly into your editor. The process took only a few seconds and the resulting file plays as smoothly as the source. There are no dropped frames and no duplicate frames. Since there is no reencoding there is no loss of quality. The only thing that's changed is the timestamps for the individual frames and the video is flagged as CFR rather than VFR. So the runtime is slightly different.

[poisondeathray]

Attached is my original 4k drone VFR video cut with Avidemux.
Maybe the blurred frames are in the original drone footage.

The frame quality differences and fluctuations are there in the original , but minor compared to the earlier file. The frequency is different (different GOP length of 30), so clearly the low quality/blur/fluctuation issue is mostly from Magix/Sony AVC encoder (GOP length 15). If you use high enough bitrate and/or better encoder it will reduce those fluctations compared to that earlier upload . But if YT is your method of distribution and viewing, those sorts of compression issues will come back

The drone does not use very good compression module and does not use b-frames; if you have the ability to set higher quality or higher bitrate, do so. It's quite soft for UHD, but looks ok at HD

Your main issue was the duplicates and there is no issue if you match up the frame rates in vegas correctly when using mp4fpsmod.

[droner]

Attached is my original 4k drone VFR video cut with Avidemux.
Maybe the blurred frames are in the original drone footage.

The frame quality differences and fluctuations are there in the original , but minor compared to the earlier file. The frequency is different (different GOP length of 30), so clearly the low quality/blur/fluctuation issue is mostly from Magix/Sony AVC encoder (GOP length 15). If you use high enough bitrate and/or better encoder it will reduce those fluctations compared to that earlier upload . But if YT is your method of distribution and viewing, those sorts of compression issues will come back

The drone does not use very good compression module and does not use b-frames; if you have the ability to set higher quality or higher bitrate, do so. It's quite soft for UHD, but looks ok at HD

Your main issue was the duplicates and there is no issue if you match up the frame rates in vegas correctly when using mp4fpsmod.

The drone camera has an option for medium to high quality video which increases the bit rate to around max 90Mb/sec
The original drone footage was recorded VBR average 75Mb/s 4k 30fps VFR.

Attached is the video using mp4fpsmod frame adjust the rendered with vegas pro using voukodec default settings.

[droner]

Been googling group of pictures GOP to get an understand of their meaning.
After reviewing the mediainfo, each of the codec's use different GOP structure.
Seem the voukodec codec uses 4 reference frames more the the MAGIX AVC rendered video and the drone source video format.
Would more reference frames ( assuming they are the I frames containing the full image) give better renders.
As the P frames only contain the difference in the next images from the previous reference frame.
Hope my understanding is correct.

[droner]

I remuxed your MKV back into an MP4 file to restore its VFR state. Then ran the script I gave earlier. The result was a CFR mp4 file that you could probably load directly into your editor. The process took only a few seconds and the resulting file plays as smoothly as the source. There are no dropped frames and no duplicate frames. Since there is no reencoding there is no loss of quality. The only thing that's changed is the timestamps for the individual frames and the video is flagged as CFR rather than VFR. So the runtime is slightly different.

Like you said the conversion i did from mp4 to avi was very quick.
Now i will have to convert back to MP4 and try the file in vegas pro.
Vegas pro wouldn't load the AVI file.
Each drone flight usually has a half a dozen video's, So if i can get the same results from the one operation using MP4FPSmod this would be easier.
Good to have 2 options to convert VFR to FFR