Twixtor - Simple guidance - Same length more frames in AE and Premier.

[Tom4]

I have Twixtor Pro 7, AE 15 and Premier 12 on a PC.


Can anybody simplify the requirements for same length, more frames for both programs?

Also, can twixtor do the above task using any initial FPS video?
Sometimes working on videos with odd frame rates like 3.65 FPS (just for example), etc.
Should I re-interpret to a standard FPS first, and what are the recognised standard FPS?

I would have thought this would be the main use by most users,
but the guidence I found just goes on and on about every other use.

Thank you

[poisondeathray]

What did you mean by "length"? Did you mean same duration ? Describe what you want in terms of frame count, fps, duration

e.g. a 10fps video with 100 frames lasts 10sec. If you have 200 frames @ 20fps, that still is 10 seconds duration . But 200 frames at 10fps means 1/2 oriingal speed and 20 seconds duration (slow motion)


I use the AE version - You don't need a standard FPS, but you need to make sure the video interpretation in the project panel, and the composition settings match. The input framerate setting in twixtor also has to match too.

"odd frame rates" - what are the sources of these videos? If you have VFR (variable framerate) video, you would probably have to convert to CFR (constant framerate) first.

[Tom4]

Sorry, I did mean duration.
So same duration, double number of frames.

I will check if any vids are VFR and change to CFR. Thanks.

Some are just odd fps, likely just hacks who edit without knowledge.
Where required, I will Reinterpret to make even numbers, make things work smoothly.

Looking for the most basic settings for task using Premier and AE.
Once I get the right advice I will try, to see if anything else is causing issues.
Reading the manual I have, its is so long and convoluted, pulling teeth would be easier :{

[poisondeathray]

I'm not sure if this is the most "simple" way - I use the AE version , and there are slightly different ways to use it (e.g. twixtor applied on another solid layer with alternate color source referencing the video), but this is the way I use


1) If the fps that AE guesses for the video is "wrong", interpret the file . Right click clip in the project panel => interpret footage => main => conform to frame rate

2) create a new composition (drag that video to make clip to the create new composition icon)

3) precompose the original layer (You can use ctrl+shift c, but I just drag the original composition in the project panel, to the make new composition icon)

4) you can rename the new composition to be organized, but in the new comp enable time remapping (right click => time => enabled time remapping ; or ctrl+alt+t)

5) If you know what fps you're going to use, change the duration of the composition settings to match it. e.g. if you know you were doing 2x , then multiple the total number frames by 2x. e.g. if your original video had 1000frames (frames 0 to 999 = 1000 frames), set it to 2000 frames. This extends the length of the composition

6) extend the length of the layer by dragging it out to the new composition length (since you enabled time remapping, this is now possible)

7) apply twixtor, make sure the Input:Frame Rate matches what the video and composition were set to. e.g. if you had 3.65fps video, set it to 3.65

8) set the speed%. At 100% it does nothing. At 50%, that would be 2x the number of frames. So in this example, right now you have 2x the frames, but the original framerate @ 3.65fps . So it plays in slow motion

9) Make a final new comp (drag the twixtor comp to new comp icon )

10) Change the frame rate of the composition settings to your goal. In this example 3.65*2 = 7.3fps

11) Enable time remapping , move (drag) the last time remap keyframe to the end and set the value to the last frame. In this example, at frame 1999 (since we have 2000 frames), the value would be 1999

I often use other tools for the last step, such as avisynth. But it depends on what you workflow is, or what you're trying to do exactly. I render out a RGB intermediate from the twixtor comp, instead of using a final comp, and use AssumeFPS() . AssumeFPS keeps the same framecount and just changes the framerate.

[Tom4]

You have no idea how much I appreciate the help
Plain English that is easily understood.

Just thanks

[Tom4]

I use the AE version - You don't need a standard FPS, but you need to make sure the video interpretation in the project panel, and the composition settings match. The input framerate setting in twixtor also has to match too.

"odd frame rates" - what are the sources of these videos? If you have VFR (variable framerate) video, you would probably have to convert to CFR (constant framerate) first.

Re: (e.g. twixtor applied on another solid layer with alternate color source referencing the video)
I do not use these features, you nailed it with answer, including a few ideas I had not considered. Excellent.



As I understand it, removing duplicates before Frame Interpolation requires vfr output.
ffmpeg -i INPUT -vf mpdecimate,setpts=N/24/TB -vsync vfr OUTPUT

Otherwise gaps will be re-filled with duplicate frames.
Lagarith would also duplicate, otherwise timing would be lost.

Is it possible to do frame Interpolation with vfr to fill a final cfr clip with unique correctly spaced frames.

With some low quality camera the frames can be large distances apart, in such case the results of any vfr Frame Interpolation solution would vary in quality throughout, no problem with that.

I gleamed something about filling with black frames, although not related, thought I would mention in case it is an idea to build on for a solution.

I could use the Adobe or Twixtor Frame Interpolation, anything that can save the result.
Ultimate solution might only use Frame Interpolation for a set distance between frames, otherwise duplicate instead.

[Tom4]

Is there any advantage to be had by spacing vfr frames to reduce duplicate places in cfr conversion.
Not sure if my logic holds or how it might best be applied?

Variable 20fps
l_l__l_l

-i INPUT -vf mpdecimate,setpts=N/5/TB -vsync vfr OUTPUT
l__l____l__l

Constant 5fps
l__l__l__l__l

Frame Interpolation for 10fps output
l_l_l_l_l_l_l_l_l_l

Save as 40fps
llllllllll

[poisondeathray]

Is it possible to do frame Interpolation with vfr to fill a final cfr clip with unique correctly spaced frames.

A VFR to CFR method with filling in the missing frames with correct temporally spaced interpolated frames instead of duplicates - I'm not aware of any fully automatic method (with other software either), but there are "manual" methods

For "auto" methods - there is a "filldrops" script and related functions in avisynth and vapoursynth , but they motion interpolate only duplicates. You can probably modify the logic to detect longer strings

There are also "manual" interpolation scripts, where you can fill the duplicates with motion interpolated frames. You specify the frame numbers of the good frames (the "goal posts" as refereces) , and you interpolate between the two points. e.g RX(101, 5) would replace 101,102,103,104,105 , by using 100 and 106 as reference points using mvtools2 as the "engine". There are modified versions that use svpflow, or rife in vapoursynth

In AE, there is manual frame interpolation script where you replace bad frames with interpolated frames as well , by marking the "bad" frames. It uses the internal "pixel motion" AE engine for interpolation . pt_FrameRestorer. You can search for it and the youtube videos

(There are significant differences in the interpolation results (quality) and settings used between algorithms, but it's another huge topic)

I gleamed something about filling with black frames, although not related, thought I would mention in case it is an idea to build on for a solution.

There is no real advantage to "black frames" vs. "pure duplicates" in terms of detection accuracy - because pure inserted duplicates in a VFR to CFR conversion are detected easily with no false positives. But a lossy compression duplicate has some threshold setting and % of error (you might miss detection, or false positive)

As I understand it, removing duplicates before Frame Interpolation requires vfr output.

In general - not necessarily . It depends on what the context is

But when you interpolate from a frame before and after that are duplicates, you get an inserted duplicate

Is there any advantage to be had by spacing vfr frames to reduce duplicate places in cfr conversion.
Not sure if my logic holds or how it might best be applied?

Variable 20fps
l_l__l_l

-i INPUT -vf mpdecimate,setpts=N/5/TB -vsync vfr OUTPUT
l__l____l__l

Constant 5fps
l__l__l__l__l

Frame Interpolation for 10fps output
l_l_l_l_l_l_l_l_l_l

Save as 40fps
llllllllll

For ffmpeg filters mpdecimate, setpts - not sure, but I can forsee problems

In the mpdecimate example - You arbitrarily selected "5" as the lowest common framerate, but if there were some sections that have say and average of 15fps, you'd be dropping "good frames". I would rather keep duplicate than drop a "good" real frame (the latter is a real motion sample)

You need to distinguish between 2 common types of VFR:

A "variable 20fps" would have gaps in the actual coded frames already. Most VFR files are timecode(timestamp) VFR. This is typical of things like phone videos. eg. It might have supposed to have been be a real 30fps CFR recording , but the average framerate might be 20fps VFR . Some sections might be 10. Using mpdecimate doesn't make sense to me to drop more frames. The key difference is display time is longer or shorter for some frames based on the timestamps. The frames are not actually encoded as duplicates/triplicates etc...

The other type of "VFR" is hardcoded duplicate VFR in CFR. The "VFR" is composed of actual inserted duplicates. e.g. if you have 15fps CFR and you duplicate the frames, you have 15fps CFR in 30 FPS. The duplicates are actually encoded. If the pattern of inserted duplicates changes you have different variable framerates in a CFR recording. Also if you take a timestamp VFR file , and convert it to CFR using "typical" methods, you now have inserted duplicates - ie. converted to "VFR in CFR" or this 2nd type.


There might be a way to code some filter based on the timestamps (timecodes) of the VFR file . In theory it should be possible

[Tom4]

Done a few experiments that support what you have raised.
It was worth a try.

Not found any easy answers, and manual fixes would end up to time consuming for return.
I have learnt a little more about the subject, so not a total waste of time.

Thanks for your advice and input

[Tom4]

Is it possible to do frame Interpolation with vfr to fill a final cfr clip with unique correctly spaced frames.

For "auto" methods - there is a "filldrops" script and related functions in avisynth and vapoursynth , but they motion interpolate only duplicates. You can probably modify the logic to detect longer strings

I found Automatically fix dups followed (eventually) by drops

# Based on script created by Didée
# Modified by John Meyer on June 29, 2011
#
# Script overview
#
# Create interpolated frames a 2x original frame rate using MVTools2
# Detect jumps 
# Create white mask at each jump point; black mask for all other frames
# Repeat each frame of original video and use mask to "choose" between original video, or motion estimated video
# Decimate exactly 50% to get back to original frame rate. 
# This decimation removes the dup frames from the original video and also the dups created by repeating each frame of original video.
# However, at each point where motion-estimated frame is inserted, no decimation occurs. Thus, if dups=drops, and the drop happens 
# within < "cycle" (TDecimate parameter) of the dup, the dup will be removed and the drop will be filled. 
# If no drops or dups occur within "cycle," then no motion estimation happens, and decimation merely gets back to original, 
# unchanged video.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Threshold for detecting jumps. Increase to catch more jumps. Should always be less than 1.0
JumpThresh = 0.74
showdot = true # true for troubleshooting; otherwise, false

#SetMTMode(5,4)
global source=AVISource( "E:\CIF Track\3200-Complete (edited).avi" ).ConvertToYV12
#SetMTMode(2)

global BlackFrame = BlankClip( source, Color=$000000 )
global WhiteFrame = BlankClip( source, Color=$FFFFFF )

super = showdot ? source.subtitle("***").MSuper(pel=2) : source.MSuper(pel=2) 
bvec  = MAnalyse(super, overlap=4, isb = true, search=4, dct=5) 
fvec  = MAnalyse(super, overlap=4, isb = false, search=4, dct=5) 
double = source.MFlowFps(super, bvec, fvec, num=60, den=1, blend=false) 

Run this on the VFR clip saved to Lagarith (Therefore a CFR clip).
Or is there a better solution for lossless conversion to CFR?
Then use Twixtor to double frame rate. Am I on track?

I do not claim to understand this method, but should be able to muddle through.

[poisondeathray]

What type of VFR (or what situation) do you have ?

A typical webcam or phone video will be timestamp(timecode) VFR - and there are many methods to convert to CFR (insert duplicate frames at roughly the correct spots, to keep the timing and sync)

That script is for both - drops and duplicates , and there are newer versions of that script in some doom9 threads. A common scenario would be a recording that drops frames, but inserts a compensatory duplicate maybe right next or, or sometimes a few frames later to keep sync. Rarely is the insert before. But avisynth itself CFR only - it does not keep track of VFR timestamps internally. ie. That script does not read VFR timecodes - the script is specific to the case where you have a CFR file ("container") , where you might have dropped frames (gaps in motion) and/or duplicate frames . It will insert new frames where there were drops, and interpolate over a duplicate, to make motion smooth. It works well in some situations. But it will not take care of long runs of duplicates, and it's based on a fixed cycle (due to reliance on tdecimate) - if you exceed the number of dups or drops in a one cycle, it won't fix them - so its not as flexible and requires some tweaking. Another option is smoothskip, but it has the same limitations with the cycle

An ideal solution would read the timestamps, and insert automatically interpolated frames instead of duplicates to achieve the desired output framerate. Mathematically it would be perfect (quality wise there will always be potential problems with interpolated frames) - but that approach would have none of this "threshold" or might be slightly wrong, or left over duplicates, or decimated real frame issues. Nobody has coded that that I'm aware.

[Tom4]

Woops

This was found with Mediainfo.
If correct, it offers a lot of important info.
Would have been helpfull when I was working on earlier.

This video would not be suitable to share here.

vid.mp4
Format : MPEG-4
Format settings : CABAC / 4 Ref Frames
Frame rate mode : Variable
Frame rate : 22.960 FPS
Minimum frame rate : 11.493 FPS *?
Maximum frame rate : 25.000 FPS

* The long still sections of many seconds tells me this info is wrong or the clip already has duplicates.
Moving through frames in good sections there are areas of 1 unique frame after another or up to 6 duplicates in row.
What is now obvious from your help, there is no one solution for vfr clips.

One for the two hard basket, but interested in what you think?

[poisondeathray]

mediainfo does not tell you anything about the actual content - what the actual encoded frames are, or if they are encoded duplicates

Frame rate : 22.960 FPS
Minimum frame rate : 11.493 FPS *?
Maximum frame rate : 25.000 FPS

This would suggest the base frame rate was 25fps . A "normal" 25fps CFR video would have a frame display time of 40ms / frame because (1/25 = 0.04 sec). The average frame rate in your example is 22.96 , so there are frame drops in sections, with compensatory timecodes where some frames are displayed longer to keep sync. Those sections have effectively a lower fps. This is timecode(timestamp) VFR . But that does not mean it cannot have actual encoded duplicates as well (it could) - ie. it could be a mixture

[Tom4]

Yes, leave it to those with more interest and knowledge.
Perhaps I will revisit the subject if a new script comes along to automate some of the manual issues.

[Tom4]

Just for the sake of sharing:

Given the MediaInfo that original was most likely 25fps CFR.
I used the following commands which gave the best results.
ffmpeg -i INPUT -vf mpdecimate -vsync vfr OUTPUT
ffmpeg -r 25000.0/1001.0 -i INPUT.mp4 -c:v copy -an OUTPUT.avi

If the video quality was good, could then apply Twixtor.
In this case, the bad sections ruined results for Interpolation.