Using various sources of various framerates in the same project

[poisondeathray]

Ok, so if the majority of music videos were shot in film in 24p, then perhaps a 24p project is what I could aim for, but still open to suggestions.

So the current episode I'm working on uses this video, which is available from youtube in a 29.97p format.

Not sure how it got to that format, but here's my download for analysis - what would once suggest I do with this?

https://www.dropbox.com/scl/fi/c20qs48qvtgw47q3wn25s/Steve-Perry-Oh-Sherrie-Video.mp4?...7sdygprwd&dl=0

If you step through the video, every 4th, 5th frame are duplicates . TDecimate() default settings are 1 in 5 decimation, that gives you 23.976p unique frames/s

The most common cause of this problem is deinterlacing a telecined source . It should have been field matched, then decimated to return the original film frames

[armyofquad]

Ok, so if the majority of music videos were shot in film in 24p, then perhaps a 24p project is what I could aim for, but still open to suggestions.

So the current episode I'm working on uses this video, which is available from youtube in a 29.97p format.

Not sure how it got to that format, but here's my download for analysis - what would once suggest I do with this?

https://www.dropbox.com/scl/fi/c20qs48qvtgw47q3wn25s/Steve-Perry-Oh-Sherrie-Video.mp4?...7sdygprwd&dl=0

If you step through the video, every 4th, 5th frame are duplicates . TDecimate() default settings are 1 in 5 decimation, that gives you 23.976p unique frames/s

The most common cause of this problem is deinterlacing a telecined source . It should have been field matched, then decimated to return the original film frames

Excellent info - how do I do this?

[poisondeathray]

Excellent info - how do I do this?

Write a script

e.g.

LSmashVideoSource("Steve Perry - Oh Sherrie (Video).mp4")
TDecimate()

That is for video only .

If you want to pass audio in the script:

aud = LSmashAudioSource("Steve Perry - Oh Sherrie (Video).mp4")

LSmashVideoSource("Steve Perry - Oh Sherrie (Video).mp4")
TDecimate()
vid=last

Audiodub(vid,aud)

[armyofquad]

Where do I use this code? Only thing I've been using that has anything like this is avspmod, but this gives me an error "there is no function named LSmashVideoSource"

[poisondeathray]

Where do I use this code? Only thing I've been using that has anything like this is avspmod, but this gives me an error "there is no function named LSmashVideoSource"

avspmod is a great script editor , that will work

LSmashVideoSource is a source filter, that can open MP4/MOV without indexing . It's part of lsmash.dll

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

[armyofquad]

Where do I use this code? Only thing I've been using that has anything like this is avspmod, but this gives me an error "there is no function named LSmashVideoSource"

avspmod is a great script editor , that will work

LSmashVideoSource is a source filter, that can open MP4/MOV without indexing . It's part of lsmash.dll

https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works/releases

Ok - the processes I've followed had me opening the scripts in vdub, but when I use vdub I have to set a compression there.

I'd rather run this as a passthrough type thing, that is, end with the same format, just with less frames.

How do I accomplish this?

[poisondeathray]

Ok - the processes I've followed had me opening the scripts in vdub, but when I use vdub I have to set a compression there.

I'd rather run this as a passthrough type thing, that is, end with the same format, just with less frames.

How do I accomplish this?

You can't really "passthrough", because avisynth frameserves uncompressed video and audio

You can't decimate an already lossy temporal compressed video (such as that YT video) without decoding first to uncompressed anyways .

In theory , if you had an I-frame format, you could discard every 5th frame, as a passthrough

So unless you're ok with uncompressed video/audio, you usually re-encode. If you're using that in another larger project, that's what I would do .

You could frameserve into vegas using avfs, but once you have many scripts, it becomes unstable. There is lots of overhead. A few should be ok

[armyofquad]

Ok - the processes I've followed had me opening the scripts in vdub, but when I use vdub I have to set a compression there.

I'd rather run this as a passthrough type thing, that is, end with the same format, just with less frames.

How do I accomplish this?

You can't really "passthrough", because avisynth frameserves uncompressed video and audio

You can't decimate an already lossy temporal compressed video (such as that one) without decoding first anyways

Ok, what should I output to then?

[poisondeathray]

Ok, what should I output to then?

An intermediate format suitable for vegas. For me and many users - that was cineform for many years.

Some recent reports of people having problems with it, however . If you're one of those having problems with cineform in vegas, maybe Prores or DNxHR .

Or high bitrate , short GOP or I frame AVC in MP4 works fine too. Actually at very high bitrates, the quality can 5-15 dB higher than even ProresXQ4444 or Cineform FS3

[armyofquad]

Ok, what should I output to then?

An intermediate format suitable for vegas. For me and many users - that was cineform for many years.

Some recent reports of people having problems with it, however . If you're one of those having problems with cineform in vegas, maybe Prores or DNxHR .

Or high bitrate , short GOP or I frame AVC in MP4 works fine too. Actually at very high bitrates, the quality can 5-15 dB higher than even ProresXQ4444 or Cineform FS3

I think Cineform will be adequate for these low quality youtube downloads. Thanks.

[armyofquad]

Ok - now comes the question of how to deal with the VHS capture that is the base of the project.

These clips are being used to replace the music video segments in a VHS recording of Friday Night Videos.

Capture wasn't done by me, found it here - https://archive.org/details/1984.05.18-kron-friday-night-videos

The mkv download is a 60p file, I'm guessing some sort of deinterlacing was done on a VHS capture to get to that point.

But being that the majority of the content is 24p film, it seems to make sense to make a project to render to 24p.

But for the material in the program between videos that I'll need to use the VHS capture for, how should I deal with the 60p? What's the best option to get that to 24p?

[poisondeathray]

Ok - now comes the question of how to deal with the VHS capture that is the base of the project.

These clips are being used to replace the music video segments in a VHS recording of Friday Night Videos.

Capture wasn't done by me, found it here - https://archive.org/details/1984.05.18-kron-friday-night-videos

The mkv download is a 60p file, I'm guessing some sort of deinterlacing was done on a VHS capture to get to that point.

But being that the majority of the content is 24p film, it seems to make sense to make a project to render to 24p.

But for the material in the program between videos that I'll need to use the VHS capture for, how should I deal with the 60p? What's the best option to get that to 24p?

Yes, it's mostly 23.976p content with a pattern of triplicates and duplicates (ie. "23.976 in 59.94") . To decimate and bring it back to 23.976, it would be TDecimate(cycleR=3, cycle=5) , for 3 in 5 decimation

However, some sections have true 59.94p content (if you go frame by frame, you see different motion each frame) - If you decimate those sections to 23.976 it will become jerky

If you leave it as 23.976 in 59.94 , it will appear as typical judder on a 60Hz display. NTSC audiences are "used" to judder

One option would be to retime the 59.94 sections as 23.976 using optical flow/ motion interpolation such as mvtools2 or RIFE . But sometimes there are artifacts

You can have to decide on the tradeoffs you want to make

[armyofquad]

Is this fixable?

https://www.dropbox.com/scl/fi/e1n6mh0mx6e9dlxo74wjy/y2mate.is-Loverboy-Queen-Of-The-B...v0schtkn1&dl=0

[armyofquad]

If a project is going to need to use both 24p and 25p sources, what options are there, and what are the pros and cons of each?

Perhaps a best compromise that favors the 24p material, one that favors the 25p material, and one thats a best overall compromise?

[Sharc]

If a project is going to need to use both 24p and 25p sources, what options are there, and what are the pros and cons of each?

Perhaps a best compromise that favors the 24p material, one that favors the 25p material, and one thats a best overall compromise?

Easiest is probably to speedup the 24fps material to 25fps. In Avisynth

AssumeFPS(25,sync_audio=true)

For music videos one may want to apply pitch correction to preserve the original pitch.

Alternatively one may apply Euro-pulldown to the 24fps stuff. This would preserve the movie duration and the audio pitch. Not sure about player support issues though.

[armyofquad]

Thanks for the suggestion.
However - any speed adjustments are completely out of the question.
Also, I'm expecting the majority of the material to be 24p, so whatever the best option for converting 25p to 24p, or converting 24p and 25p to another format, would be a better direction to go in I'm thinking.

[Sharc]

Euro-pulldown doesn't change the speed.

ChangeFPS(25) preserves the speed as well but would insert frames (duplicates or blends).
See also here for more options:
https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/fps.html

If you reduce 25fps material to 24fps you would have to drop frames.

More complicated are framerate conversions by means of temporal interpolation. Prone to artifacts though.

[poisondeathray]

For Loverboy vid, I'd try double blend removal. The pattern is 3 "good" frame, 2 blends

LSmashVideoSource(y2mate.is - Loverboy Queen Of The Broken Hearts Official Video -hWSLgpQGw3I-480pp-1711828654.mp4")
srestore(frate=23.976, omode="pp2")
preroll(24)
tdecimate()

Some scenes aren't "fixed" that well, but others look ok

[armyofquad]

For Loverboy vid, I'd try double blend removal. The pattern is 3 "good" frame, 2 blends

LSmashVideoSource(y2mate.is - Loverboy Queen Of The Broken Hearts Official Video -hWSLgpQGw3I-480pp-1711828654.mp4")
srestore(frate=23.976, omode="pp2")
preroll(24)
tdecimate()

Some scenes aren't "fixed" that well, but others look ok

Thanks for that. Kinda sad that the "official" uploads to youtube get things wrong like this - the TV rips I can understand, but the official ones....


Anyways, I'm missing some pre-requisites. I determined I needed to download Srestore.avsi, but now I'm getting an error "Script error: there is no function named 'average'", and it's calling on line 53 in Srestore - must be a pre-requisite that I can't seem to find, and with a term as generic as "average", googling isn't quite finding what I need.

[poisondeathray]

Anyways, I'm missing some pre-requisites. I determined I needed to download Srestore.avsi, but now I'm getting an error "Script error: there is no function named 'average'", and it's calling on line 53 in Srestore - must be a pre-requisite that I can't seem to find, and with a term as generic as "average", googling isn't quite finding what I need.

Prerequisites and links to them should be listed on the avisynth wiki , including links to average which is listed on the srestore page

http://avisynth.nl/index.php/Srestore

[armyofquad]

My plug-ins aren't playing nice with each other, not sure how I'm supposed to get a stable environment with this scenario.

I found average.

Now I get the error "Average: All clips must be have same or greater number of frames as the first one"

Yes, that's right, must be have same.

Googling led me to a suggestion of a version of smdegrain being a culprit, went to a "current" site and downloaded a copy, got a syntax error.

I'm on 64bit versions of these plug-ins.


What do I need to do to get stability and get these to all play nicely?

[armyofquad]

Ok, managed to find newer versions of plug-ins and yet new pre-requisites, and got that script working. Thanks again.


Moving right along, here's another "official" youtube post - thoughts on this one?


https://www.dropbox.com/scl/fi/nr95ka96px03onhcyqdvd/Thompson-Twins-Doctor-Doctor-Vide...iwerfo2tc&dl=0

[Sharc]

Nothing particularly wrong with it. 29.97fps progressive IMO.
Little sharpening and framerateconverted (using RIFE) to 23.976fps if this is what you have in mind:

[armyofquad]

Nothing particularly wrong with it. 29.97fps progressive IMO.
Little sharpening and framerateconverted (using RIFE) to 23.976fps if this is what you have in mind:

Cool, thanks.

Is one able to determine what the original framerate was? I was finding most of these videos were originally shot at 24 or 25, so the 29.97p seemed an odd rate, like the result of a previous conversion?

What steps did you take to do the conversion, so I could repeat this myself on future clips?

[Sharc]

Is one able to determine what the original framerate was?

Not really for clean conversions. For poor conversions one can often conclude on the specific artifacts (blends, repetition patterns ...) what the original framerate has most likely been.

What steps did you take to do the conversion, so I could repeat this myself on future clips?

Installation of RIFE:
- Copy the rife.dll in Avisynth's plugins64+ folder
- Copy the models in a folder 'models' in Avisynth's pugins64+ folder

Avisynth script which I used for that video:

a=LWLibavAudioSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
v=LWLibavVideoSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
Audiodub(a,v)
awarpsharp() #sharpen

# frame rate conversion with RIFE:
v_RGB=z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f") # RIFE works with RGB floats
v_RIFE=v_RGB.RIFE(gpu_thread=1, model=25, fps_num=24000, fps_den=1001, sc=true, sc_threshold=0.3) # motion interpolate to 24000/1001 fps, model=25 (find the 'best' model by trying, minimizing artifacts)
v_YV12=v_RIFE.z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l") # convert back to YV12
return v_YV12

[armyofquad]

Is one able to determine what the original framerate was?

Not really for clean conversions. For poor conversions one can often conclude on the specific artifacts (blends, repetition patterns ...) what the original framerate has most likely been.

What steps did you take to do the conversion, so I could repeat this myself on future clips?

Installation of RIFE:
- Copy the rife.dll in Avisynth's plugins64+ folder
- Copy the models in a folder 'models' in Avisynth's pugins64+ folder

Avisynth script which I used for that video:

a=LWLibavAudioSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
v=LWLibavVideoSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
Audiodub(a,v)
awarpsharp() #sharpen

# frame rate conversion with RIFE:
v_RGB=z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f") # RIFE works with RGB floats
v_RIFE=v_RGB.RIFE(gpu_thread=1, model=25, fps_num=24000, fps_den=1001, sc=true, sc_threshold=0.3) # motion interpolate to 24000/1001 fps, model=25 (find the 'best' model by trying, minimizing artifacts)
v_YV12=v_RIFE.z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l") # convert back to YV12
return v_YV12

Thanks for this.

The uploaded sample seems to cut early, so I'm trying to run this myself to correct that.

However, I keep running into "RIFE: failed to load model"

I've not used rife before, I did install it and the pre-requisites - but these various scripts from various sources are a hot mess of garbage getting to work.

How can I get a consistent stable install of RIFE?

How can I get a stable situation where it isn't a hot mess every time someone recommends another plug-in?

[armyofquad]

Is one able to determine what the original framerate was?

Not really for clean conversions. For poor conversions one can often conclude on the specific artifacts (blends, repetition patterns ...) what the original framerate has most likely been.

What steps did you take to do the conversion, so I could repeat this myself on future clips?

Installation of RIFE:
- Copy the rife.dll in Avisynth's plugins64+ folder
- Copy the models in a folder 'models' in Avisynth's pugins64+ folder

Avisynth script which I used for that video:

a=LWLibavAudioSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
v=LWLibavVideoSource("Thompson Twins - Doctor! Doctor! (Video).mp4")
Audiodub(a,v)
awarpsharp() #sharpen

# frame rate conversion with RIFE:
v_RGB=z_ConvertFormat(pixel_type="RGBPS", colorspace_op="709:709:709:l=>rgb:709:709:f") # RIFE works with RGB floats
v_RIFE=v_RGB.RIFE(gpu_thread=1, model=25, fps_num=24000, fps_den=1001, sc=true, sc_threshold=0.3) # motion interpolate to 24000/1001 fps, model=25 (find the 'best' model by trying, minimizing artifacts)
v_YV12=v_RIFE.z_ConvertFormat(pixel_type="YUV420P8", colorspace_op="rgb:709:709:f=>709:709:709:l") # convert back to YV12
return v_YV12

You reference copying models into a folder called models, and the RIFE documentation references models.


What are models? Where are models? Where do they get them? How are end users expected to know this without any actual explanation about this? (not asking to be a dick, honestly trying to understand this - is this being poorly documented? Or is there actually something listed on the page that I'm overlooking?)

[Sharc]

The uploaded sample seems to cut early, ....

I cut it for demo. Here the full length:

[armyofquad]

The uploaded sample seems to cut early, ....

I cut it for demo. Here the full length:

Thanks, that solves the immediate issue. Appreciate it.

[Sharc]

You reference copying models into a folder called models, and the RIFE documentation references models.

What are models? Where are models? Where do they get them? How are end users expected to know this without any actual explanation about this? (not asking to be a dick, honestly trying to understand this - is this being poorly documented? Or is there actually something listed on the page that I'm overlooking?)

From the link which I provided you have to download the models (file models.7z) as well, unzip it and you'll get a pile of folders. Create a new folder 'models' in the plugins64+ folder to copy these folders you got from unzipping models.7z into.

So
- download models.7z
- create an empty new folder 'models' in the plugins64+ folder
- unzip the content of models.7z into that newly created folder