"Grok" sudden jump in the motion.

[dg3duy]

Hey everyone, hope you can help me out with this because I've been going at it for a while and can't figure it out.

I have some videos generated with Grok that when played back look like they're skipping, as if a few frames are just missing at that point. It's not that the audio cuts out or the video freezes completely — it's more like a sudden jump in the motion. It happens randomly, not always in the same spot, and the weird thing is it doesn't happen on every video I generate — some come out perfectly fine, others have these skips.

The videos are MP4, 416x752 (vertical format), 24fps. There are no duplicate or frozen frames — the motion just jumps from one point to another with no transition, as if those frames simply don't exist in the file.

I tried using RIFE in Hybrid to see if it could reconstruct the missing frames through interpolation, but I'm not sure if that's even the right approach for this kind of problem.

Has anyone dealt with something like this with AI-generated video? Is there a tool or workflow you'd recommend for detecting and reconstructing truly missing frames? Any help is appreciated, thanks!

https://drive.google.com/file/d/16j3PSsayDw7Q4xj5nM4JN9Mfckze7TjE/view?usp=sharing

[Selur]

Background seem fine, the hair doesn't move naturally,...
From my point of view: There are no frames missing, the model simply has no concept of hair movement or doesn't use that knowledge.

Cu Selur

[dg3duy]

Background seem fine, the hair doesn't move naturally,...
From my point of view: There are no frames missing, the model simply has no concept of hair movement or doesn't use that knowledge.

Cu Selur

Thank you very much for taking the time to reply. In the background, you can see the “abrupt” jump in some frames. I'll leave a video where you can see which frames are the jumps.
https://drive.google.com/file/d/1Uajp_YbxzIBxF2grt8Ve944LBIOieFFo/view?usp=sharing

Thank you very much and best regards!

[Selur]

You can try replacing single frames with interpolations, but I doubt you will find an automated way to fix those frames,..

[dg3duy]

You can try replacing single frames with interpolations, but I doubt you will find an automated way to fix those frames,..

That's what I wanted to hear, from someone with experience on the subject. You gave me the idea of working on it by sections rather than the whole video. Just where you see the jump, apply interpolation “we'll see if it works or not” for now it's just a theory.
Thank you very much!

[dg3duy]

@selur Taking advantage of the situation, what tool do you recommend for interpolation? Your intuition never fails.

[Selur]

RIFE would is probably the best you can use atm.

[dg3duy]

RIFE would is probably the best you can use atm.

Your help is excellent! Thank you very much.

[jagabo]

Using RIFE:

function InsertFramesRIFE(clip Source, int N, int "X")
{
  # N is the frame number in Source where frames will be inserted
  # X is total number of frames to insert
  # e.g. InsertFramesRIFE(101, 3) would insert 3 frames between 100 and 101 reference points for interpolation
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=12, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N,0)
}



LWLibavVideoSource("Video.mp4", cache=false, prefer_hw=2) 


InsertFramesRIFE(133)
InsertFramesRIFE(109)
InsertFramesRIFE(85)
InsertFramesRIFE(61)

Subtitle("RIFE") was used here to make it easy to identify which frames were interpolated with RIFE. Remove that line for final production.

[dg3duy]

Using RIFE:

function InsertFramesRIFE(clip Source, int N, int "X")
{
  # N is the frame number in Source where frames will be inserted
  # X is total number of frames to replace
  # e.g. InsertFramesRIFE(101, 3) would insert 3 frames between 100 and 101 reference points for interpolation
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=12, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N,0)
}



LWLibavVideoSource("Video.mp4", cache=false, prefer_hw=2) 


InsertFramesRIFE(133)
InsertFramesRIFE(109)
InsertFramesRIFE(85)
InsertFramesRIFE(61)

Subtitle("RIFE") was used here to make it easy to identify which frames were interpolated with RIFE. Remove that line for final production.

I see it and I can't believe the final result! I'm blown away with excitement! It looks really excellent! I don't have words to thank you enough. There are good people in this forum.

[Selur]

Alternatively Hybrid allows using ReplaceMultipleFrames and FillDuplicateFrames in 'Replace' mode to simply specify which frames to replace with interpolations. (both offer using, mvtools, rife and svp for interpolation)

Cu Selur

[dg3duy]

Alternatively Hybrid allows using ReplaceMultipleFrames and FillDuplicateFrames in 'Replace' mode to simply specify which frames to replace with interpolations. (both offer using, mvtools, rife and svp for interpolation)

Cu Selur

Impressive! Thank you very much for the information! Now I can continue without feeling the constant frustration of noticing jumps in the video. I'm very excited.

[martaorplq]

That kind of jump usually isn’t “missing frames” in the file. With AI-generated video the model sometimes produces inconsistent motion between frames, so when it’s encoded at 24fps it looks like a skip even though every frame exists. That’s why tools that look for duplicates or freezes don’t flag anything.

Your idea of using RIFE is actually reasonable. Frame interpolation can smooth those jumps by generating in-between frames. Another approach is to first convert the clip to a higher framerate (like 48 or 60fps) with interpolation using tools such as SVP Flow or Flowframes, then re-encode. Some people also run a light motion smoothing pass in DaVinci Resolve or VapourSynth.

If the jumps are severe, though, it usually means the generation itself wasn’t temporally stable, and the cleanest fix is regenerating that segment.

[dg3duy]

That kind of jump usually isn’t “missing frames” in the file. With AI-generated video the model sometimes produces inconsistent motion between frames, so when it’s encoded at 24fps it looks like a skip even though every frame exists. That’s why tools that look for duplicates or freezes don’t flag anything.

Your idea of using RIFE is actually reasonable. Frame interpolation can smooth those jumps by generating in-between frames. Another approach is to first convert the clip to a higher framerate (like 48 or 60fps) with interpolation using tools such as SVP Flow or Flowframes, then re-encode. Some people also run a light motion smoothing pass in DaVinci Resolve or VapourSynth.

If the jumps are severe, though, it usually means the generation itself wasn’t temporally stable, and the cleanest fix is regenerating that segment.

Hello,

Thank you very much for your message and for taking the time to help.

I tested the other methods you suggested, such as converting the clip to a higher framerate (like 48 or 60 fps) with interpolation using tools like Flowframes and also DaVinci Resolve, but unfortunately they did not solve the problem.

So far, the only solution that has worked correctly for me is using RIFE.

I really appreciate your help and the suggestions you shared. Thank you again for your time.

Best regards.

[Hochbach]

Using RIFE:

function InsertFramesRIFE(clip Source, int N, int "X")
{
  # N is the frame number in Source where frames will be inserted
  # X is total number of frames to replace
  # e.g. InsertFramesRIFE(101, 3) would insert 3 frames between 100 and 101 reference points for interpolation
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=12, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N,0)
}



LWLibavVideoSource("Video.mp4", cache=false, prefer_hw=2) 


InsertFramesRIFE(133)
InsertFramesRIFE(109)
InsertFramesRIFE(85)
InsertFramesRIFE(61)

Subtitle("RIFE") was used here to make it easy to identify which frames were interpolated with RIFE. Remove that line for final production.

I see it and I can't believe the final result! I'm blown away with excitement! It looks really excellent! I don't have words to thank you enough. There are good people in this forum.

technology works wonders

[jagabo]

RIFE has the best motion interpolation for most material (in AviSynth). But it's not perfect and can sometimes generate gross distortions.

Note that the "missing" frames occur every 24 frames in the supplied sample. This suggest GROK generated a 25 fps video but discarded every 25th frame to make a 24 fps video.

[Hochbach]

RIFE is impressive for motion interpolation

[Sharc]

Using RIFE:

function InsertFramesRIFE(clip Source, int N, int "X")
{
  # N is the frame number in Source where frames will be inserted
  # X is total number of frames to insert
  # e.g. InsertFramesRIFE(101, 3) would insert 3 frames between 100 and 101 reference points for interpolation
  #
  # requires YV12, YV16, or YV24 input, outputs same as input

  X = default(x, 1)

  Trim(Source, N-1, 1)+Trim(Source, N, 1)
  z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f")
  Rife(gpu_thread=1, model=12, factor_num=X+1, factor_den=1, sc=false)
  z_ConvertFormat(pixel_type=Source.pixeltype, colorspace_op="rgb:709:709:f=>709:709:709:l") 

  AssumeFPS(Source.FrameRateNumerator, Source.FrameRateDenominator)
  Trim(1,X)
Subtitle("RIFE")

  Source.trim(0,N-1) ++ last ++ Source.trim(N,0)
}



LWLibavVideoSource("Video.mp4", cache=false, prefer_hw=2) 


InsertFramesRIFE(133)
InsertFramesRIFE(109)
InsertFramesRIFE(85)
InsertFramesRIFE(61)

Subtitle("RIFE") was used here to make it easy to identify which frames were interpolated with RIFE. Remove that line for final production.

@jagabo
Shouldn't the function be called like

InsertFramesRIFE(source,133)
InsertFramesRIFE(source,109)
InsertFramesRIFE(source,85)
InsertFramesRIFE(source,61)

or alternatively:

source
InsertFramesRIFE(133)
InsertFramesRIFE(109)
InsertFramesRIFE(85)
InsertFramesRIFE(61)

If we skip the 'source' the subseqent frame numbers get shifted by the previously inserted frame(s), no?