Is it possible to shift a video with half a frame?

[Bencuri]

I am working on a 25p progressive video, and to sync with the audio correctly the video should be cut at around 1/3 time, and the second half should be shifted backwards with half a frame. Is it possible to do it somehow? If you asked how the video got longer, the audio is an overdubbing and there is a point where it becomes to be off. The audio cannot be cut because it will be audible and disturbing.

[timtape]

The audio can be digitally stretched or shrunk. Also short audio sections can be removed, down to one sample in length. I found it easier to initially stretch the audio to be slightly slower than the video. Then small slices of audio can be removed at strategic points to maintain sync. The sync doesnt have to be 100% accurate, just close enough that the errors are not noticeable to the viewer. I have done this on music videos uploaded to YT where picture and sound were initially well out of sync. Can you upload a section to illustrate?

[poisondeathray]

It's generally better to manipulate the audio.


You would need higher temporal resolution if you wanted to manipulate the video. To shift the video a half frame back in time at 25p - you'd interpolate to 50p, select every odd frame (the interpolated ones). You risk artifacts due to interpolation depending on the content and algorithm used, possibly significantly worse than manipulating the audio. If going this route I'd try RIFE first

Schematically it would look like this, where the original 25p video is ABCD. "AB", "BC" etc.. are the interpolated "inbetween" frames . AB would indicate the video sample between original A and B (ie. "half a frame" ahead of A)

A    B    C    D  [25p]
A AB B BC C CD D  [50p]
AB   BC   CD      [25p]

[Bencuri]

Thanks! I have tried changing the audio, but it is noticeable. You cannot just shrink it in time, the first part of the video is in sync, the second half is a quarter/half a frame off. But I have a second version where only a few frames are off throughout the video. I have a script somewhere for interpolation, I will try to use it, along with the scheme you suggest. But in case only a small portion was interpolated, how should the modified section and the next unchanged section be joined? For example, from your scheme above, should the new order be: AB BC CD and continues on D, or between CD and D there should still be something else?

[poisondeathray]

But in case only a small portion was interpolated, how should the modified section and the next unchanged section be joined? For example, from your scheme above, should the new order be: AB BC CD and continues on D, or between CD and D there should still be something else?

The schematic carries on as long as you want until the next unchanged section . For example if the 1st frame of the unchanged section was "G"

AB BC CD DE EF FG | G

The FG to G transition point will be 1/2 the distance of what it should be (In original recording, it was F G , but FG is the midpoint)

For some people it might be easier to visualize in a NLE . You can overlay an interpolated section and shift it

If instead you are a quarter frame off , you would need higher resolution again - you'd need to interpolate to a higher frame rate

[Bencuri]

I have this code, but I noticed it was used to convert 60fps to 50fps. Can you help how to modify it to make 25 fps to 50? Or it does that automatically in case I add (50,1) as parameter?

#requires MVTools
function SmoothFPS2(clip source, int num, int den) { 
super = MSuper(source, pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(source, super, backward_3, forward_3, num, den, mode=0)
} 

DirectShowSource("1z.MP4", seek=false) 
SmoothFPS2(50,1)

[poisondeathray]

I have this code, but I noticed it was used to convert 60fps to 50fps. Can you help how to modify it to make 25 fps to 50? Or it does that automatically in case I add (50,1) as parameter?

#requires MVTools
function SmoothFPS2(clip source, int num, int den) { 
super = MSuper(source, pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(source, super, backward_3, forward_3, num, den, mode=0)
} 

DirectShowSource("1z.MP4", seek=false) 
SmoothFPS2(50,1)

I would avoid using DirectShowSource (not necessarily frame accurate) , but if the source filter returns 25fps correctly then SmoothFPS2(50,1) should be correct

But SmoothFPS2 quality (using mvtools2) is going to be worse than using RIFE for the majority of cases, but much faster to process

[Bencuri]

https://github.com/Asd-g/AviSynthPlus-RIFE/releases/tag/1.2.3

I cannot check it on the avisynth wiki because the site seems to be down, but I gess you mean the one above.

Can you suggest a script for this one? The FPS of the source is 25p progressive as mentioned, if more details needed here they are (the scan type is interpreted wrongly, we have verified that here many times).

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Format settings, GOP : M=3, N=12
Codec ID : 27
Duration : 4 min 23 s
Bit rate mode : Variable
Bit rate : 11.5 Mb/s
Maximum bit rate : 12.0 Mb/s
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan type, store method : Separated fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.297
Stream size : 363 MiB (94%)

[poisondeathray]

https://github.com/Asd-g/AviSynthPlus-RIFE/releases/tag/1.2.3

I cannot check it on the avisynth wiki because the site seems to be down, but I gess you mean the one above.

Yes

The FPS of the source is 25p progressive as mentioned, if more details needed here they are (the scan type is interpreted wrongly, we have verified that here many times).

Can you suggest a script for this one?

Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First

I'll assume that it's 25p content. You will see other types of artifacts otherwise


You can try different models out, some could be better on things like guitar strings and fingers. I used 44 in this example

LSmashVideoSource("1z.MP4")
AssumeFrameBased()
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f", use_props=0)
RIFE(fps_num=50, fps_den=1, model=44)
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:709:709:f=>709:709:709:l", use_props=0)

[Bencuri]

Thanks! I will try both methods, and we'll see how useful they are. There were occasions when SmoothFPS could also create a good looking video.

One last question: if I import the 25p video and the 50p interpolated version into 1 project in Vegas, and place the created frames above the problematic ones manually to replace them, can that be a solution, or Vegas will mess up everything that way? There is a setting: Resampling for such mixed projects, it is on Smart Resample by default. Should I switch it to something else, there is also Forced Resample or Disable Resample. Or it doesn't matter, and I should avoid to bring the two together in one project because the order of frame dropping in the end when exporting in 25p will be totally unpredictable?

[poisondeathray]

Thanks! I will try both methods, and we'll see how useful they are. There were occasions when SmoothFPS could also create a good looking video.

Ok, but 99% of the time RIFE will be better, and probably 50% of the time a lot better

One last question: if I import the 25p video and the 50p interpolated version into 1 project in Vegas, and place the created frames above the problematic ones manually to replace them, can that be a solution, or Vegas will mess up everything that way? There is a setting: Resampling for such mixed projects, it is on Smart Resample by default. Should I switch it to something else, there is also Forced Resample or Disable Resample. Or it doesn't matter, and I should avoid to bring the two together in one project because the order of frame dropping in the end when exporting in 25p will be totally unpredictable?

To do it that way, where you have access to all 50p frames, the timeline has to be 50p for you to "see" them all, disable resample (so you get duplicates on the 25p video)

Or, you can use SelectOdd() in the scripts , and make the "50p" video 25p with only the interpolated frames present and use a 25p project - similar to the schematic in post 3. That way is probably easier

[Bencuri]

Thanks a lot!

Okay, we'll se if this can help. I hope it can, because until I buy an 50p camcorder this issue will return time by time.