Mixed video of 25fps and 30fps - which is it best to convert to and how?

[Dave2ic]

I previously had a Sony NEX-3 camera which had MP4 video of 1280x720 at 30fps.

Now I have a Sony NEX-F3, and presumably because I live in a PAL region the MP4 video option of 1440x1080 is locked to 25fps.

If I have video in both frame rates that I want to edit together, is it best to convert the 25fps to 30fps or 30fps to 25fps? Does it matter which way you go to get the best/smoothest video? What is the best tool to achieve this?

I'm using MP4 and not AVCHD because I find it easier to work with on a PC (demuxing/muxing, joining/splitting etc). The MP4 is at a lower bitrate and resolution (1440 vs 1920 wide), but it looks pretty good and is a bit more economical on disk space - and it is just family videos I'm doing after all.

Thanks in advance for any assistance.

[Cornucopia]

You should go with where you live. PAL region = 25fps. Make all your stuff compatible with that.

Smoothness depends alot on the algorithm and/or method used, not just the app(s). AVISynth is very good and free, but takes some learning.

Couple of choices:
Speedup/slowdown ok?
If not what is worse to you - jitteryness or blended frames?

Scott

[Dave2ic]

Thanks, Scott.

I have used AVISynth (a long time ago) for deinterlacing some Mini DV video that I had, so I'm sure I can manage it if I have some advise on the plugins to use and examples of the script to follow.

I wouldn't have thought speedup/slowdown would have been an option worth considering. It's only family videos we're talking about but I would imagine slowing everything from 30 fps to 25 fps would look unnatural. Then of course you'd have to change the audio length, then presumably compensate for the corresponding pitch change.

I can give it a try though. I imagine from what you are saying is that this would result in the best video output as you're retaining the same video frames and not having to re-encode anything - the video just displays at a different speed.

When you say 'jitteryness or blended frames', I assume you mean that going 30 fps => 25 fps means dropping every 6th frame (jitteryness) and 25 fps => 30 fps means blending every 5th and 6th frame (blending frames).

Jitteryness sounds bad, but I guess either option is in essence playing the video either sped up or slowed down but compensation every 1/5 or 1/6 of a second with a blended or skipped frame. Blended frames sounds at least that it will be smoother.

Perhaps if you can point me to the best ways to do all these options in AVI Synth - speed up/slowdown, jitteryness and blended frames - I can try them out and see what works best for me.

I'm a bit disappointed that I've ended up with a camera that has locked me into 25 fps when any other camera I've had has been 30 fps. If it's not going to be possible to achieve good results converting framerates, I'm just wondering if I'm better off selling this camera and trying to get the version that has 30 fps.

Thanks for any advice.

[poisondeathray]

I'm a bit disappointed that I've ended up with a camera that has locked me into 25 fps when any other camera I've had has been 30 fps. If it's not going to be possible to achieve good results converting framerates, I'm just wondering if I'm better off selling this camera and trying to get the version that has 30 fps.

Aren't you in a 50Hz region ? New Zealand ? You're better off getting 50Hz equipment, so 25 FPS . There is no good way for 25<=>30FPS conversions, they all have side effects because the numbers aren't evenly divisible

A simple slowdown is too much (the difference between 30 and 25 is too great for audio slowdown) . You have 3 basic options for 30=>25

1) Dropping Frames. changefps(25). Will give a choppy result

FFVideoSource("30FPS_Video.mp4")
ChangeFPS(25)

2) Blending Frames. convertfps(25). This will give a blurry result but might appear marginally smoother.

FFVideoSource("30FPS_Video.mp4")
ConvertFPS(25)

3) Motion Interpolation by generating new "inbetween" frames to 150fps, evenly dividing by 6 to get 25. Since the frames are evenly spaced, it will give the smoothest results. The "negatives" are longer processing times, and edge morphing artifacts (on some types of footage it's more visible, on others it's not noticable)

There are several mvtools2 based functions that might yield slightly better or worse results, I'll list the one MFlowFPS

source = FFVideoSource("30FPS_Video.mp4")

super = source.MSuper(pel=2)
backward_vec = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vec = MAnalyse(super, overlap=4, isb = false, search=3)
source.MFlowFps(super, backward_vec, forward_vec, blend=false, num=25, den=1)

[Dave2ic]

Aren't you in a 50Hz region ? New Zealand ? You're better off getting 50Hz equipment, so 25 FPS

Yes I am, but all our TV's/DVD players etc handle NTSC or PAL, so 30fps is not a problem. I don't care whether I have 25fps or 30fps video, but it is much easier just to deal with a single frame rate rather than have to convert one to the other.

3) Motion Interpolation by generating new "inbetween" frames to 150fps, evenly dividing by 6 to get 25. Since the frames are evenly spaced, it will give the smoothest results. The "negatives" are longer processing times, and edge morphing artifacts (on some types of footage it's more visible, on others it's not noticable)

Thanks very much - this sounds like the way to go. I will give it a try and report back my results. I don't mind long processing times to acheive a better result.

[poisondeathray]

Aren't you in a 50Hz region ? New Zealand ? You're better off getting 50Hz equipment, so 25 FPS

Yes I am, but all our TV's/DVD players etc handle NTSC or PAL, so 30fps is not a problem. I don't care whether I have 25fps or 30fps video, but it is much easier just to deal with a single frame rate rather than have to convert one to the other.

But your HDTV 's refresh rate is at 50Hz (or a multiple, some LCD's are 250Hz, 500Hz or some even multiple of 50) . 30p content on a 50Hz display won't look as smooth as 25p content on a 50Hz display for the same reasons earlier - they are not evenly divisible and frames won't be evenly spaced - some frames will be displayed longer than others so there will be a cadence "judder". (of course real 50p would look perfect on it)

[Dave2ic]

source = FFVideoSource("30FPS_Video.mp4")

super = source.MSuper(pel=2)
backward_vec = MAnalyse(super, overlap=4, isb = true, search=3)
forward_vec = MAnalyse(super, overlap=4, isb = false, search=3)
source.MFlowFps(super, backward_vec, forward_vec, blend=false, num=25, den=1)

I tried this code but when I opened the script in VirtualDub all I got was "There is no function names "FFVideoSource"".

I have mvtools2 and ffms2 plugins in the AVISynth plugins folder and even tried preceeding the code here with: LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ffms2.dll"

Have got a missing plugin, or a wrong version of a plugin? What am I missing?

Thanks.

[poisondeathray]

Yes, it's a plugin not included with standard avisynth install

http://code.google.com/p/ffmpegsource/

[Dave2ic]

Yes, it's a plugin not included with standard avisynth install
http://code.google.com/p/ffmpegsource/

Thanks a lot - got it sorted and done some testing.

1. Dropping frames
Give best image quality for each frame - but of course results in slightly jerky video due to the dropped frames.

2. Blending frames
Smoother than dropping frames, but results in noticable ghosting which I found unpleasant.

3. Motion Interpolation
Slow to process but gives generally very good results. I did however find that you can get really bad 'tearing' of the video in very fast panning scenes - I guess this is where there is so much difference between 2 frames that there is not enough information to generate intermediate frames. Are there other options/variables adjustment that may be good to try to see if they improve this?

I imagine that the best results could be achieved by a combination of Motion Interpolation and either Dropping Frames or Blending Frames. Motion Interpolation gives the best results when the frame differences aren't too much, but when the frame differences are too great like in fast panning it would be nice to be able to switch to Dropping Frames or Blending Frames so you don't get that tearing of the picture. I don't know if anyone has ever done such a script - or if it's even possible!

Thanks very much for assistance in all this poisondeathray - I've achieved what I started out this thread wanting to know.

[poisondeathray]

I did however find that you can get really bad 'tearing' of the video in very fast panning scenes - I guess this is where there is so much difference between 2 frames that there is not enough information to generate intermediate frames.

Yes, certain types of content are more predisposed to the artifacts, like fast movement, rotational movements, non linear movements, blocking movements (objects crossing front of others) - with all these scenarios , the motion vectors are more difficult to predict , and the edges of objects are no as well delineated (so you get these weird edge morphing artifacts)

Are there other options/variables adjustment that may be good to try to see if they improve this?

Yes, there are dozens and dozens of settings where you can tweak. I hate to say this, but have a look at the manual

There are other filters you can try as well that have slight variations and maybe slightly better or worse - SVPFlow, InterFrame, SmoothFPS2 to name a few

In my experience , the difference is very small despite expermenting with all the settings. What causes mflowfps to "fail" are the same set of conditions that cause the other filters to fail (and even other programs like twixtor, nuke, after effects , etc)

I imagine that the best results could be achieved by a combination of Motion Interpolation and either Dropping Frames or Blending Frames. Motion Interpolation gives the best results when the frame differences aren't too much, but when the frame differences are too great like in fast panning it would be nice to be able to switch to Dropping Frames or Blending Frames so you don't get that tearing of the picture. I don't know if anyone has ever done such a script - or if it's even possible!

I don't know if this is possible to do "automatically" . You might have to combine some manual trims() with different filters. Or it would be easier in a NLE, where you might mix & match easier

There are better ways in compositing applications, what's done is foreground and background masks to delineate shape borders in something like twixtor pro . In this way you can improve the motion estimation and vector quality - but there is quite a bit of manual work with mask/rotowork

FYI, NLE's will do either (1) or (2) above for their sampling . If you have frame blend off, they will act like (1) , if you have frame blend on , they will resample like (2) . Most do not offer method (3)

The bottom line is there is no good way to do these types of framerate conversions . The "best" way is to shoot the ideal framerate in the first place

[smrpix]

Don't force yourself into a "one size fits all" solution. If you're making an assembly of mostly "old" material use 30 fps. If it's mostly new use 25.

Some scenes can be sped up or slowed down for a 1:1 frame rate without being noticeable -- say a countryside shot out of a car window.

[smrpix]

Also, your camera can record at either framerate.

--oops. maybe not. If you use AVCHD instead of MP4 is it switchable?

[poisondeathray]

Don't force yourself into a "one size fits all" solution. If you're making an assembly of mostly "old" material use 30 fps. If it's mostly new use 25.

Some scenes can be sped up or slowed down for a 1:1 frame rate without being noticeable -- say a countryside shot out of a car window.

Yes, those are some good ideas .



You can do creative editing and incorporate those shots , when there is no speech it will work well for those types of shots , or you can overlay other types of audio . It all depends on your composition

For completeness, if you wanted 1:1 slowdown (all real frames) in avisynth , you use AssumeFPS(25) . This would be equivalent in a NLE to interpreting the footage as 25p - essentially the clip is played slower speed

[Dave2ic]

Also, your camera can record at either framerate.

--oops. maybe not. If you use AVCHD instead of MP4 is it switchable?

Unfortunately, no - you only get 50i or 25p. I'm sure it would be easy enough for Sony to allow you to select 25 or 30 fps in a firmware update if they wanted to. It's so annoying that the NEX-3 I had recorded at 30fps, but the NEX-F3 that I now have will only do 25fps.

[poisondeathray]

Just curious - was the NEX-3 bought in a NZ store ? I know some camera models don't offer "PAL" framerates, is there a 25p model available ?

[Dave2ic]

I don't know if this is possible to do "automatically" . You might have to combine some manual trims() with different filters. Or it would be easier in a NLE, where you might mix & match easier

Sorry if this is a silly question, but what's an NLE?

[poisondeathray]

I don't know if this is possible to do "automatically" . You might have to combine some manual trims() with different filters. Or it would be easier in a NLE, where you might mix & match easier

Sorry if this is a silly question, but what's an NLE?

NLE = non linear editor

eg. sony vegas pro, premiere pro, final cut pro, avid media composer etc...


It's much easier to edit with them, when you have a timeline, multiple tracks, proper editing tools

Avisynth is all script driven, it's harder to edit things in (avspmod is the "closest" thing, but it's not even close to a real NLE for editing tasks)

[Dave2ic]

Just curious - was the NEX-3 bought in a NZ store ? I know some camera models don't offer "PAL" framerates, is there a 25p model available ?

Yes, it was purchased in New Zealand. This was after I read reviews that it did 30fps - I only subsequently found out that the frame rate is different for different regions - 25fps for PAL regions, 30fps for NTSC.

It would be great if I could have imported an NEX-F3 from the US just to get te 30fps frame rate. But the cost of doing so - not to mention the loss I'd make on selling my current camera - makes that an unrealistic option.

[Dave2ic]

Don't force yourself into a "one size fits all" solution. If you're making an assembly of mostly "old" material use 30 fps. If it's mostly new use 25.

Some scenes can be sped up or slowed down for a 1:1 frame rate without being noticeable -- say a countryside shot out of a car window.

Good points and duly noted!

[poisondeathray]

If you wanted to go 25=>30 , just change the numbers to 30 . For mflowfps, where the line says num=25, change it to 30. That assumes you mean 30.0 FPS. Standard FPS for NTSC is actually 29.97 (or 30000/1001 exactly) . You would enter num= 30000 , den=1001 for that

When you did your tests, did you view the results on the TV or computer monitor ?

I ask because most computer monitors run at 60Hz , even in PAL regions . But your "PAL" HDTV runs at some multiple of 50Hz . 25p will look fine on the TV, but 30p will have a slight judder . Vice versa on a 60Hz monitor . Some people are very sensitive to this judder, others can't even tell the difference

[Dave2ic]

If you wanted to go 25=>30 , just change the numbers to 30 . For mflowfps, where the line says num=25, change it to 30. That assumes you mean 30.0 FPS. Standard FPS for NTSC is actually 29.97 (or 30000/1001 exactly) . You would enter num= 30000 , den=1001 for that

Is one way better than the other or does 25=>30 lead to as many artifact problems as 30=>25?

When you did your tests, did you view the results on the TV or computer monitor ?

I'm doing all my tests on a monitor, which is 60Hz as you say.

Pretty much all my viewing is on my PC, so 25fps or 30fps all seems to be fine.

At the moment I'm just contemplating how much of an issue this is going to be for me. It's basically just family videos I'm doing (my kids), and if I think about it I'm unlikely to be editing together video of different frame rates unless in the future I do some sort of video of the kids growing up. And thanks to you I've got some tools and knowledge now to deal with the frame rate issue if I need to. I've already got some old 25fps MiniDV footage, so it's not the first time I've face a change in frame rates.

I'm very happy with the NEX-F3 in all other respects, so I guess I shouldn't be too hung up on one small aspect like the frame rate of the video.

Excuse the rant - I just want to get this all clear in my head so I don't find myself 12 months down the track with all this 25fps video and regretting that it wasn't 30fps.

[smrpix]

So it comes down to how you would rather spend your time -- enjoying your kids or adding artifacts to your videos that aren't there in the first place.

[poisondeathray]

Is one way better than the other or does 25=>30 lead to as many artifact problems as 30=>25?

Same types of artifacts. Situations that predispose to causing artifacts are the same regardless

But when start with higher native framerates , the interpolation is generally better - you have more data represented per second, and the distance travelled is less for objects, so there is less room for interpolation error. You won't be able to tell between 25 and 30, I'm talking about higher ones like 60, 120