Speed up a video with AVISynth

[Alwyn]

I am dabbling with Super 8 cine I have shot with my Canon HV20 (shutter speed 1/50, frame rate 25fps PAL) and recorded as LAGS AVIs. Yes yes yes I am aware of the opinions of others on this technique but that's the one I'm using and I'll be happy with the end result if I can get this speed sorted out.

The results, obviously, are a bit of a mess regarding duplicated frames, and I have applied TFM and TDecimate to remove them. The resultant file is 16.67 frames per second.

I'm a bit confused by the next step. What I think I should do is get the speed back up to native 18 frames per second, but I can't find a filter that will do that.

I would then use something like Framerate converter or RIFE (haven't gone there yet!) to get the frame rate up to 25fps so I can put it into my NLE without it (my NLE) chucking in random frames willynilly to go from 16.67 to 25. I think I should be able to do better with AVISynth.

I have scoured the forum and also Doom but can't find what I'm after.

My question therefore is, how do I change the "speed" of my video, as opposed to the frame rate?

Thanks very much for any input.

[davexnet]

Just use

Assumefps(18)

this will do the speedup you asked for

[Alwyn]

Thanks Dave!

[killerteengohan]

Thanks Dave!

That will indeed speed up the video like you asked, but that method will make it go out of sync with audio the longer it plays if you use that method. If you are not using any audio, then it's no big deal and a quick easy solution for what you asked.

[Alwyn]

Cheers Killer, I'll just stretch/destretch the audio as necessary to fit.

In the end, I didn't continue using Assume 18 because it made my video way too fast. I'm still trying to work out why!

[Sharc]

The solution I see would be remove the duplicates and then interpolate (framerateconverter, RIFE) to the new framerate. Firstly make sure that you did remove true duplictes only. As 18fps (acc. Super8 cine) is way too fast (as you wrote) I would speculate that you removed too many frames which possibly were no true duplicates.

Added:
For recovering the unique 18fps frames out of your 25fps oversampled video, did you try:

TDecimate(Cycle=25,CycleR=7)

It will ditch the 7 most similar frames from a sequence ("window") of 25 frames.
http://avisynth.nl/index.php/TIVTC/TDecimate

[DB83]

TBH (and I might be having a senior moment here) I can not see where 'duplicate' frames come in to the argument.

Maybe you should expand on the initial 'capture' process. If you have projected the super8 on to a screen/wall and then filmed that at your declared settings how can duplicate frames happen ? Many moons ago I was grabbing super8 via a mirror-box. No 'duplicate' frames but there was always a projector shutter issue which caused 'flashing' since the displayed footage was a different speed to the videod one.


That 'flashing' could be corrected via a vdub filter but you still had a 25 fps video.


Maybe you should provide a short sample of the original 'capture'

[Sharc]

I can not see where 'duplicate' frames come in to the argument.

As I understood it, the OP projected the film at 18fps (a new picture appears every 55.6ms on screen) and re-filmed it at 25fps (creating a frame every 40 ms). Hence some film pictures will inevitably be duplicated (sampled twice) by this "capture" process at a higher framerate.
Yes, a sample may clarify.

Added:
Due to the aperture (exposure) duration of his video camera he might also see occasional blends of 2 adjacent frames of the original 18fps pictures, because a film frame may change during the time the videocam's shutter is open. Plus other issues .....

Added2:
Explaining where the duplicates come from (simplified)

[poisondeathray]

The results, obviously, are a bit of a mess regarding duplicated frames, and I have applied TFM and TDecimate to remove them. The resultant file is 16.67 frames per second.

I'm a bit confused by the next step. What I think I should do is get the speed back up to native 18 frames per second, but I can't find a filter that will do that.

Was it projected at 18/1 FPS, or was it a vari-speed projector purposely set to 16.67 fps ? You wouldn't change the speed if you projected at normal speed.

16.67 fps implies you are dropping "good" frames if it was a normal speed projection

TDecimate(cycler=7, cycle=25) should give you 18/1 FPS instead of 16.67fps


(16.667 varispeed is sometimes used in a PAL 25i or 50p setup, because this gives you triplicates which are "easy" to decimate . 16.667*3 = 50) .=>Then you would speed it back up to 18 (reversing the varispeed slowdown)

[johnmeyer]

The OP pretty clearly stated that he transferred his Super 8 movie film by shooting it with his Canon HV20 set for PAL recording speed. Since he made no mention of a modified projector, and since he acknowledged that his technique is not approved (certainly not by me!), I think it is a fair assumption that he just filmed the output of his projector. Since he did not use a high shutter speed on his HV20, he will get all sorts of problems including blended frames and also wild exposure variations depending on whether the projector shutter was closed during a portion of the exposure for each frame. Some video frames will be much darker because the shutter happened to be closed during part of the exposure time.

Previous posts about sound sync are pointless because this is Super 8 movie film, and this format does not support sound.

Getting the playback speed to be correct is, by far, the least of the OP's worries. The real problem will be flicker and blended frames.

The OP may find some help in this lengthy thread over at doom9.org:

The power of AVISynth salvaging "botched" transfers of old 8mm films to DVD.

[edit]Here is the botched video given as example in that thread. It will show others what the problem looks like.

https://vimeo.com/24361245

[jagabo]

Yes, I would guess there's no audio too (there was some super 8 film with audio though). But if there is, and you are handling it in AviSynth, AssumeFPS() can adjust the audio too. Use the sync_audio=true argument:

http://avisynth.nl/index.php/AssumeFPS

It syncs the audio by changing the sample rate. You'll usually want to resample back to standard audio rate with SSRC. The AssumeFPS page has an example of that.

[Alwyn]

Sorry for the delay, folks. My head is spinning with this stuff. I can't replicate the dupe frames that I was removing with TFM and TDecimate, but I'm pretty sure that I got 16.666 (and also 8.3333) by adjusting the parameters for TDecimate. I must admit I still don't fully understand what cycle and cycleR signify.

The issue I'm having is when I remove the dupes, I'm ending up with a slightly slower video, but when I apply AssumeFPS it then goes too fast. I have tried Framerateconverter as well, which works really well for a few seconds, super smooth, but then coughs badly for a few frames before going smooth again. I don't understand most of the options for FRC.

Anyway, here's one of the original clips I am working on. I've left it upside down in they interests of "purity". It has plenty of blended frames but no dupes as far as I can tell. It looks pretty ugly frame by frame, but when playing it in realtime with no restoration, I'm now thinking it isn't too bad.

From a Canon HV20 direct into the lens, 1440x1080 HDMI livestream > Startech USB3HDCAP > with AmarecTV. I slowed the projector down with a voltage regulator to reduce rolling.

Ancillary question: do the black borders affect operation of AVISynth filters and should I crop them away before processing?

[Alwyn]

@johnmeyer, to address a couple of your comments:

Since he did not use a high shutter speed on his HV20, he will get all sorts of problems including blended frames and also wild exposure variations depending on whether the projector shutter was closed during a portion of the exposure for each frame. Some video frames will be much darker because the shutter happened to be closed during part of the exposure time.

The higher the shutter speed on my camera, the worse the rolling and black frames got. The reality (for me at least) is the slower the better. I have used 6, 12, 25, and 50.

Previous posts about sound sync are pointless because this is Super 8 movie film, and this format does not support sound.

Yes it does. I have a few rolls here with sound.

The real problem will be flicker and blended frames.

Flicker isn't a major issue for me; I can control the projector speed and Neat video clears up most of the residual. The blended frames, yes, I am interested in fixing and returning to the proper speed.

Here is the botched video given as example in that thread. It will show others what the problem looks like.

Half the problem with that "botchup" is the globe. Using modern lighting methods, that is a non-issue.

[jagabo]

Remove CycleR frames, preferably duplicates, from every group of Cycle frames.

But you want to remove blended frames, not duplicate frames. Use SRestore(frate=N) where N is the frame rate you want.

But what you really want is to digitize properly. I agree that the proper procedure is to capture at a high frame rate, then discard the black and partially black or dark frames, leaving only unblended frames.

[Alwyn]

Remove CycleR frames, preferably duplicates) from every group of Cycle frames.

Using an example of CycleR of 5 and Cycle of 6, does that mean frames 7-11, 18-22 will be removed, or does it pick what frames to remove? That's what I don't understand in the context of the dupe frames, which don't appear to be in a set 5-in-6 pattern.

I agree that the proper procedure is to capture at a high frame rate, then discard the black and partially black or dark frames, leaving only unblended frames.

Fair enough, that wasn't clear from John's comment.

[poisondeathray]

Remove CycleR frames, preferably duplicates) from every group of Cycle frames.

Using an example of CycleR of 5 and Cycle of 6, does that mean frames 7-11, 18-22 will be removed, or does it pick what frames to remove? That's what I don't understand in the context of the dupe frames, which don't appear to be in a set 5-in-6 pattern.

It picks using metrics between the most similar frames in the cycle. The algorithm is adaptive, not a set cycle like every nth frame. It takes into account scene changes, and there are many advanced parameters (look at the readme)

5 in 6 would be you keep 1 out of every 6 frames. 5 most similar frames are dropped out of 6. If you started with 25fps, you end up with 4.167fps

[johnmeyer]

The higher the shutter speed on my camera, the worse the rolling and black frames got. The reality (for me at least) is the slower the better. I have used 6, 12, 25, and 50.

Actually, you want those black frames.

Over a decade ago I shared some of the secrets of my design for using a shutterless projector to make perfect frame-accurate film to video transfers. The key was using TFM and TDecimate with some strange settings so they removed the duplicates. I won't describe the process here, but I outlined it in this post:

Recover image from shutterless 16mm projector capture

You have a very nice-looking capture, except for all the blends and duplicates. Here's what you need to do.

Re-capture using a high shutter speed, at least 1/500 of a second. Then, look at the resulting capture, frame-by-frame. I think you will find that some frames are completely black because the shutter is closed. Using the high shutter speed eliminates blends.

The blends are your big problem and are impossible to fully remove; by contrast, getting rid of black frames is trivial. The high shutter speed eliminates those blends. You need to make one pass with AVISynth to remove the black frames. I have several scripts which do this. Here is one:


Finding individual "bad" frames in video; save frame number; or repair

You'll then need to look at the resulting video and see if there are any duplicates. You then simply use TFM followed by TDecimate to remove the duplicates. That is what TVITC (which contains TFM and TDecimate) is designed to do. You set Cycle for the number of frames (the group) you want to evaluate, and CycleR for the number of frames you want to delete from each group. For instance, to recover 18 fps film from 30 fps video you use Cycle= 5 and CycleR = 2. If you divide 5 into 30 there are six "Cycles" in every second of video and there will therefore be six CycleR or 6*2 = 12 frames that will be deleted. 30-12 = 18. You could also use Cycle = 30 and CycleR = 12 and get the same thing. Larger Cycle/CycleR sizes makes the script run much slower, but will do a better job on video where the pattern doesn't repeat exactly within a shorter Cycle period.

For your situation, I have no idea what Cycle/CycleR numbers to choose because I don't know the actual speed of your projector. Your camera is obviously taking 50 frames per second (if you can set it to 50 fps progressive) or 25 fps. Use 50 fps if you can: the faster the frame rate the better, and the shorter the shutter speed the better. You probably won't get a perfectly repeating pattern because your projector is not synchronized to your camera. Therefore you will want to use pretty high Cycle and CycleR values in order to have as few misses as possible when a dup happens at a Cycle boundary.

It is probably possible to recover something from what you have already captured, and that thread I linked to earlier is the way to do that. Forget about the hot spot in the middle of that sample provided in that post. It has nothing to do with the problem you are trying to solve. When he used the word "botched," he was talking about the problem you have created for yourself, and therefore his script should work for you.

However, it would be better to capture using the method I described. His script works pretty well (I have used it), but it can never give you the perfect result that I think you can get if you follow my instructions.

[jagabo]

Remove CycleR frames, preferably duplicates) from every group of Cycle frames.

Using an example of CycleR of 5 and Cycle of 6, does that mean frames 7-11, 18-22 will be removed, or does it pick what frames to remove? That's what I don't understand in the context of the dupe frames, which don't appear to be in a set 5-in-6 pattern.

It removes duplicates. If there are no duplicates it removes near duplicates, if there are no near duplicates it removes the most similar frame, etc.

[Alwyn]

I do appreciate all your inputs, thank you.

Re black frames, I did a couple of tests, but cannot achieve any completely non-black frames, as per the attached, with any combination of shutter speed, play speed or capture frames per second. Even with the projector speed synched to my camera (no rolling), there is always a portion of a black bar on the screen.

Re TFM and TDecimate, I did some experiments with different Cycle and CycleR values (from your descriptions, I now understand what those numbers actually mean ). While I was able to whittle down the blended frames to around 1 in 25, the frame rate came way down, and I came back to the original issue: AssumeFPS(18) sped up the video way too much. As soon as I use Framerateconverter, the video turns into a mess. There are obviously lots of variables for either FRC or AssumeFPS to make them work correctly.

I've downloaded John's script from his link so will experiment with it.

[johnmeyer]

If you have Super 8 film and not regular film, then is always plays back at 18 fps. If AssumeFPS makes the video play too fast, then you have deleted too many frames and need to adjust your settings.

You say you are downloading "John's script," but if you are trying to get good results with your original capture, then you need to download the script that another person developed (the "botched" script).

I looked at your 50 fps video and something is most definitely not right. This cannot possibly be your original capture because you get five frames of motion followed by five duplicated frames. You must have modified this video before uploading.

I may not be able to help any further because I now realize I actually don't know what you are doing.

[Sharc]

There are obviously lots of variables for either FRC or AssumeFPS to make them work correctly.

AssumeFPS does neither add nor remove any frames. It simply plays the existing frames at the stated fps, like adjusting the speed of a projector. There is nothing to make it work correctly. It always works correctly.

[Alwyn]

However, it would be better to capture using the method I described. His script works pretty well (I have used it), but it can never give you the perfect result that I think you can get if you follow my instructions.

I thought your script on the Doom9 link you gave was what you were pointing me to. FPP's is for bad MPEG transfers. I'll have another look at Freds.

I looked at your 50 fps video and something is most definitely not right.

Yes, I don't know what happened there, perhaps a VDub filter was applied unknowingly, or I may have done a full-processing re-encode for the snippet by mistake. However, that's not my point though. I was demonstrating that there are no completely clean frames; every frame, when I capture at 500 or 1000/sec as you suggested, has at least a bit of black, so I don't see how I can "remove the black frames and then TFM/TDecimate the remainder". I've attached a "correct" capture showing the black-in-every-frame phenomenon.

Thanks for your previous comments though, they help me understand.

[Alwyn]

AssumeFPS does neither add nor remove any frames. It simply plays the existing frames at the stated fps, like adjusting the speed of a projector. There is nothing to make it work correctly. It always works correctly

OK, I think I have it now. IMO, it's not named logically. "FPS" is normally frames per second, not frame rate/play speed. It should be called AssumeSpeed.

On the other hand, FrameRateConverter doesn't actually change the rate, it changes the number of frames in each second. The video still plays at the same "speed".

I think!

[Sharc]

And there is ChangeFPS. It drops or repeats (duplicates) existing frames in order to achieve the stated FPS. It will neither speed up nor slow down. It will keep the duration of a clip and hence preserve video/audio sync. Usually the motion will be choppy due to the duplicated or dropped frames.

And there is ConvertFPS in Blend or Switch mode. For details see the Avisynth docs.

FramerateConverter or RIFE synthesizes new frames by temporal interpolation of adjacent frames (instead of simply duplicating or dropping as ChangeFPS does), avoiding choppy playback. The quality of the interpolated pictures depend for example on the source (motion, foreground vs background) and on the size of the temporal gaps which have to be interpolated.

P.S. Can you perhaps remove the shutter of your projector (or keep it open all the time) and repeat the capture experiment?

[Alwyn]

Can you perhaps remove the shutter of your projector (or keep it open all the time) and repeat the capture experiment?

Aha, of course! John was talking about his shutterless rig on Doom9. Let me see if I can do that.

[Sharc]

Is the light source of your projector stable (driven from a stabilized DC source), or does it flicker with the mains frequency (50Hz) which creates beats with the camera's 50 fields per second, which would increasingly average out with longer camera exposure times? Wild speculations ....

[johnmeyer]

Can you perhaps remove the shutter of your projector (or keep it open all the time) and repeat the capture experiment?

Aha, of course! John was talking about his shutterless rig on Doom9. Let me see if I can do that.

Depending on your projector, removing the shutter can be a chore. On my Eiki 16mm it was trivial but on my two Bell & Howell 8mm/Super 8 projectors, it would be almost impossible.

You still need to figure out what was going on with the high shutter speed capture you sent. Nothing about that capture made any sense. I say this because as the shutter blade advanced across the screen, I could see image movement behind the blade. This is impossible because a projector is designed so that pulldown to the next frame only happens when the shutter is fully closed.

[Alwyn]

Thanks, I'll have a look.

[Alwyn]

Is the light source of your projector stable (driven from a stabilized DC source)

Ceiling LED through a 24v DC driver and dimmer.

[jagabo]

Is the light source of your projector stable (driven from a stabilized DC source)

Ceiling LED through a 24v DC driver and dimmer.

That's probably pulse width modulated.

https://en.wikipedia.org/wiki/Pulse-width_modulation

See the "duty cycle" section. The LED is turning off and on at a high rate. That would explain why you never see a full frame.