Is it possible to shift a video with half a frame?

[Bencuri]

I am working on a 25p progressive video, and to sync with the audio correctly the video should be cut at around 1/3 time, and the second half should be shifted backwards with half a frame. Is it possible to do it somehow? If you asked how the video got longer, the audio is an overdubbing and there is a point where it becomes to be off. The audio cannot be cut because it will be audible and disturbing.

[timtape]

The audio can be digitally stretched or shrunk. Also short audio sections can be removed, down to one sample in length. I found it easier to initially stretch the audio to be slightly slower than the video. Then small slices of audio can be removed at strategic points to maintain sync. The sync doesnt have to be 100% accurate, just close enough that the errors are not noticeable to the viewer. I have done this on music videos uploaded to YT where picture and sound were initially well out of sync. Can you upload a section to illustrate?

[poisondeathray]

It's generally better to manipulate the audio.


You would need higher temporal resolution if you wanted to manipulate the video. To shift the video a half frame back in time at 25p - you'd interpolate to 50p, select every odd frame (the interpolated ones). You risk artifacts due to interpolation depending on the content and algorithm used, possibly significantly worse than manipulating the audio. If going this route I'd try RIFE first

Schematically it would look like this, where the original 25p video is ABCD. "AB", "BC" etc.. are the interpolated "inbetween" frames . AB would indicate the video sample between original A and B (ie. "half a frame" ahead of A)

A    B    C    D  [25p]
A AB B BC C CD D  [50p]
AB   BC   CD      [25p]

[Bencuri]

Thanks! I have tried changing the audio, but it is noticeable. You cannot just shrink it in time, the first part of the video is in sync, the second half is a quarter/half a frame off. But I have a second version where only a few frames are off throughout the video. I have a script somewhere for interpolation, I will try to use it, along with the scheme you suggest. But in case only a small portion was interpolated, how should the modified section and the next unchanged section be joined? For example, from your scheme above, should the new order be: AB BC CD and continues on D, or between CD and D there should still be something else?

[poisondeathray]

But in case only a small portion was interpolated, how should the modified section and the next unchanged section be joined? For example, from your scheme above, should the new order be: AB BC CD and continues on D, or between CD and D there should still be something else?

The schematic carries on as long as you want until the next unchanged section . For example if the 1st frame of the unchanged section was "G"

AB BC CD DE EF FG | G

The FG to G transition point will be 1/2 the distance of what it should be (In original recording, it was F G , but FG is the midpoint)

For some people it might be easier to visualize in a NLE . You can overlay an interpolated section and shift it

If instead you are a quarter frame off , you would need higher resolution again - you'd need to interpolate to a higher frame rate

[Bencuri]

I have this code, but I noticed it was used to convert 60fps to 50fps. Can you help how to modify it to make 25 fps to 50? Or it does that automatically in case I add (50,1) as parameter?

#requires MVTools
function SmoothFPS2(clip source, int num, int den) { 
super = MSuper(source, pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(source, super, backward_3, forward_3, num, den, mode=0)
} 

DirectShowSource("1z.MP4", seek=false) 
SmoothFPS2(50,1)

[poisondeathray]

I have this code, but I noticed it was used to convert 60fps to 50fps. Can you help how to modify it to make 25 fps to 50? Or it does that automatically in case I add (50,1) as parameter?

#requires MVTools
function SmoothFPS2(clip source, int num, int den) { 
super = MSuper(source, pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(source, super, backward_3, forward_3, num, den, mode=0)
} 

DirectShowSource("1z.MP4", seek=false) 
SmoothFPS2(50,1)

I would avoid using DirectShowSource (not necessarily frame accurate) , but if the source filter returns 25fps correctly then SmoothFPS2(50,1) should be correct

But SmoothFPS2 quality (using mvtools2) is going to be worse than using RIFE for the majority of cases, but much faster to process

[Bencuri]

https://github.com/Asd-g/AviSynthPlus-RIFE/releases/tag/1.2.3

I cannot check it on the avisynth wiki because the site seems to be down, but I gess you mean the one above.

Can you suggest a script for this one? The FPS of the source is 25p progressive as mentioned, if more details needed here they are (the scan type is interpreted wrongly, we have verified that here many times).

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Format settings, GOP : M=3, N=12
Codec ID : 27
Duration : 4 min 23 s
Bit rate mode : Variable
Bit rate : 11.5 Mb/s
Maximum bit rate : 12.0 Mb/s
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan type, store method : Separated fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.297
Stream size : 363 MiB (94%)

[poisondeathray]

https://github.com/Asd-g/AviSynthPlus-RIFE/releases/tag/1.2.3

I cannot check it on the avisynth wiki because the site seems to be down, but I gess you mean the one above.

Yes

The FPS of the source is 25p progressive as mentioned, if more details needed here they are (the scan type is interpreted wrongly, we have verified that here many times).

Can you suggest a script for this one?

Scan type                                : Interlaced
Scan type, store method                  : Separated fields
Scan order                               : Top Field First

I'll assume that it's 25p content. You will see other types of artifacts otherwise


You can try different models out, some could be better on things like guitar strings and fingers. I used 44 in this example

LSmashVideoSource("1z.MP4")
AssumeFrameBased()
z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f", use_props=0)
RIFE(fps_num=50, fps_den=1, model=44)
z_ConvertFormat(pixel_type="YV12", colorspace_op="rgb:709:709:f=>709:709:709:l", use_props=0)

[Bencuri]

Thanks! I will try both methods, and we'll see how useful they are. There were occasions when SmoothFPS could also create a good looking video.

One last question: if I import the 25p video and the 50p interpolated version into 1 project in Vegas, and place the created frames above the problematic ones manually to replace them, can that be a solution, or Vegas will mess up everything that way? There is a setting: Resampling for such mixed projects, it is on Smart Resample by default. Should I switch it to something else, there is also Forced Resample or Disable Resample. Or it doesn't matter, and I should avoid to bring the two together in one project because the order of frame dropping in the end when exporting in 25p will be totally unpredictable?

[poisondeathray]

Thanks! I will try both methods, and we'll see how useful they are. There were occasions when SmoothFPS could also create a good looking video.

Ok, but 99% of the time RIFE will be better, and probably 50% of the time a lot better

One last question: if I import the 25p video and the 50p interpolated version into 1 project in Vegas, and place the created frames above the problematic ones manually to replace them, can that be a solution, or Vegas will mess up everything that way? There is a setting: Resampling for such mixed projects, it is on Smart Resample by default. Should I switch it to something else, there is also Forced Resample or Disable Resample. Or it doesn't matter, and I should avoid to bring the two together in one project because the order of frame dropping in the end when exporting in 25p will be totally unpredictable?

To do it that way, where you have access to all 50p frames, the timeline has to be 50p for you to "see" them all, disable resample (so you get duplicates on the 25p video)

Or, you can use SelectOdd() in the scripts , and make the "50p" video 25p with only the interpolated frames present and use a 25p project - similar to the schematic in post 3. That way is probably easier

[Bencuri]

Thanks a lot!

Okay, we'll se if this can help. I hope it can, because until I buy an 50p camcorder this issue will return time by time.

[Bencuri]

I have just downloaded RIFE to try it, but noticed in the Requirements in says: Vulkan device. What does this mean actually? I have Intel HD4000 in my laptop I am not sure that is acceptable.

[poisondeathray]

I have just downloaded RIFE to try it, but noticed in the Requirements in says: Vulkan device. What does this mean actually? I have Intel HD4000 in my laptop I am not sure that is acceptable.

It means a GPU that supports vulkan , not sure if your does . You can check with GPU-z , or vulkaninfo. But I suspect you might have problems with GPU memory amount on a 1920x1080 frame

https://www.techpowerup.com/gpuz/
https://docs.vulkan.org/guide/latest/checking_for_support.html

[Bencuri]

For the first check it seems it is only supported on Linux in my case.

But we'll see. SmoothFPS is still there as a B option.

However, I ran into a problem that in spite of that the video info says 16:9, when I preview the script in AVSPmod, the video opens as 4:3. It is 1440x1080 though. I have not worked with this frame size yet, what do I need to do to make it render and look correctly? I want to use x264.exe. Do I just need to add --width 1920 --height 1080 in x264.exe, and that's it, or is there a way for avisynth to open it properly? The video preview shows an incorrect sizing.

[poisondeathray]

For encoding, if you want it same as original 1440x1080 displaying as 1920x1080 - you add --sar 64:45 , or just resize it 1920x1080 in the script

For avspmod, you can resize to 1920x1080 temporarily just for the display. There might be some other display functions in newer versions of avspmod, not sure

[Bencuri]

I did not want to struggle with Rife tonight, will check it tomorrow. Tried the SmoothFPS this time, what's your opinion? I couldn't catch any serious artifact in this. Maybe only around the guitar pick it looks as if there was a black ghosting or patch.

By the way, from the MV tools pack I can select the MSVC version for an i5-3210M, right?

[poisondeathray]

I did not want to struggle with Rife tonight, will check it tomorrow. Tried the SmoothFPS this time, what's your opinion? I couldn't catch any serious artifact in this. Maybe only around the guitar pick it looks as if there was a black ghosting or patch.

For smoothfps2/mvtools2 - it produces more blobby ghosting/morphing artifacts around the fingers and stings or around edges of moving objects, like pick, thumb. Sometimes as the hands go over strings, they become distorted and twisted at weird angles. RIFE produces cleaner results, fewer artifacts

By the way, from the MV tools pack I can select the MSVC version for an i5-3210M, right?

yes

[Bencuri]

The problem is I get the error message: There is no function named Rife. However, the dll is in the plugins library. I put it into the plugins64+ directory and running AVSPmod 64 bit. Maybe that is the problem?

[poisondeathray]

The problem is I get the error message: There is no function named Rife. However, the dll is in the plugins library. I put it into the plugins64+ directory and running AVSPmod 64 bit. Maybe that is the problem?

The .dll should autoload, but you can try LoadPlugin("PATH\RIFE.dll")

Did you meet the other requirements on that page in terms of AVS+ version, and Microsoft VisualC++ Redistributable Package 2022 ?