More Video Source Handling Questions..

[alcOre]

First let me start by saying thank you so much to you guys that keep helping me out through this learning experience.

Anyways, I have some more sources that I'm not sure what to do with. I also have a few questions. One, I've tried using SRestore on other sources that were similar to my other source (the music video that someone told me to use SRestore(FRate=24) because it was converted from a 24 fps source.) I took another source that I counted the unique frames and it was 24, just like my other source. I tried the same script on that one, and it came out jittery. I don't remember which source it was so I can't post it. I guess my question is when/how is one supposed to use SRestore?

Also, If tvs are 60 hertz in the US, and thats the reason 24 fps content is converted to 29.97i video so that it equals the tv's hertz, how do tvs display 23.976p content? Does the player or the tv add frames at playback?

And I have a few different sources here that I don't know how to properly handle. I'll post sample clips. (These are all music videos.)

1st Source/Sample : http://www.mediafire.com/file/mgjgwgmyzqj/Eminem-RealSlimShady-Sample.m2v

2nd Source/Sample : http://www.mediafire.com/file/ytjzixitl1m/Enur-Calabria-Sample.m2v

3rd Source/Sample : http://www.mediafire.com/file/gzonqj42jam/Katerine-ShutYourMouth-Sample.m2v

4th Source/Sample : http://rapidshare.com/files/206706281/KP-IKAG-Sample.m2v.html

Two of the sources seem to have field blending (at least that's what I think it is.) And a couple of them seem to have had frames added by something like ChangeFPS that I don't know how to undo.

Sorry there's soo many different sources. I've learned a lot but I still have some more to learn. Thanks in advance to any help with these sources.

[manono]

Also, If tvs are 60 hertz in the US, and thats the reason 24 fps content is converted to 29.97i video so that it equals the tv's hertz, how do tvs display 23.976p content? Does the player or the tv add frames at playback?

That's what 3:2 pulldown is all about. The player takes that 23.976fps and converts it to 59.94 fields per second, to match the 59.94 Hz. Read this, the section a little ways down titled, "NTSC Telecine (Transfer of 24fps Film to 60Hz NTSC TV)":

http://www.hometheaterhifi.com/volume_7_4/dvd-benchmark-part-5-progressive-10-2000.html

1. Not field blended, but converted to PAL by adding one frame in every 25 in the form of 2 fields in every 50, but it was done in a very unusual way so that :

TFM()
TDecimate(Mode=2,Rate=24)#or TDecimate(Mode=2,Cycle=25,CycleR=1)

doesn't work as it should. Both leave duplicate frames and drop unique frames. Even though it blends some frames together, I had my best luck with:

Yadif(Order=1,Mode=1)
BlendBob()
Tdecimate(Mode=2,Rate=24)

About all you can do is play around with different methods. It's very messed up.

2. Standard field blend from PAL2NTSC. This works well:

Yadif(Order=1,Mode=1)
RePAL()#or Srestore

3. Not field blended but converted from PAL in a much more sensible way:

TFM()
TDecimate(Mode=0,Cycle=6,CycleR=1)

4. Can't download. It says "The page cannot be found".

[alcOre]

1) Thanks for the reply manano...yah I know what the 3:2 pulldown is. But what i'm confused about is how the new progressive tvs display a progressive source picture. Like a bluray to a 1080p plasma. What's going on there? isnt the 24pfps -> 23.976p -> 29.97i (through 3:2 pulldown) for displaying an interlaced picture? What happens with progressive sources and tvs in order to display a progressive picture? EDIT : Nevermind. Read your posted article. Explained what I wanted to know about progressive to progressive tvs. I still would like to know how computer displays handle different things but hey, that's one question out of the way.

2) 4th sample not working on mediafire for some reason. Tried re-upping multiple times. Changed it to rapidshare if you want to look at that sample manano (fixed link in original post.)

3) About this :

2. Standard field blend from PAL2NTSC. This works well:

Yadif(Order=1,Mode=1)
RePAL()#or Srestore

RePAL() is the same as SRestore()? I thought SRestore() would take it and turn it to like 11.88 fps or something like that.

4) From what I gather, yadifmod is better than yadif correct? (Or am I comparing apples to oranges.) Can I just replace Yadif with Yadifmod in those scripts you posted? Or are the paramaters that Yadifmod supports different?

[guns1inger]

3:2/3:2 pulldown is used to allow progressive material (usually from film sources) play back on interlaced devices (older televisions). How newer TVs handle it comes down to a combination of the player and the TV. Many players have a progressive output option which will send a 23.976 fps progressive signal to the TV. A 1080p TV should upscale this and display is as 24 fps progressive. If you don't use the progressive option on the player then the TV gets a 29.970 fps interlaced signal, and displays it as such.

Remember - 1080p means it can accept and play back a 1080p signal natively. It does not mean that everything it displays is 1080p.

[alcOre]

Yah...I know that much. What my question is comes down to hertz. An older tv set is getting the correct number of fields to match its hertz..ala 29.97i is basically 30i...which is 1 frame/hertz for a 60hertz tv if my understanding is correct. But how to the new tvs (and computer monitors for that matter) display the video when its not getting a signal that exactly matches the hertz of the screen? (I'm currently reading that posted article by manano to see if that answers my question.)

[manono]

With a progressive display the 23.976fps video gets played back in the same 3:2 pattern, but by frames rather than by fields. Where each letter represents a frame:

AAA BB CCC DD EEE FF

What the new 120Hz TVs accomplish is to remove that slight 3:2 jerkiness by playing each frame 5 times (if originally 23.976fps):

AAAAA BBBBB CCCCC DDDDD

[manono]

The 4th sample is just like the 2nd:

Yadif(Order=1,Mode=1)
RePAL()#or Srestore

I thought SRestore() would take it and turn it to like 11.88 fps or something like that.

You're supposed to bob it first, and you set the final framerate with the FRate parameter. For NTSC stuff the default final framerate is 25fps.

[alcOre]

Oh...I see. (about the bobbing.) Stupid question then...what's the best bobber to use? (and what's the difference between a regular bob and a smart bobber?)

Christ...I hate to keep asking stupid questions but could you post a complete script for bobbing then using srestore for the 2nd/4th source? Thanks...

[manono]

A regular bob, like the one built into AviSynth, interpolates everything off of each field. A smart bobber leaves the static parts at full resolution and only interpolates the motion parts. The best bobber, hands down, is TempGaussMC. It's also absurdly slow. Here's a bobber comparison that's been floating around this site for awhile, one originally supplied by Soopafresh:

https://forum.videohelp.com/images/guides/p1934885/stockholma_0-520_q3_yadif_mvbobmod_t...mca4_tdtmm.avi

Here's the info on TempGaussMC:

http://avisynth.org/mediawiki/TempGaussMC

Here's a script using SRestore for the 2nd and 4th samples:

Yadif(Mode=1,Order=1)
SRestore()#or SRestore(FRate=24.975) or RePAL

[alcOre]

Here's a script using SRestore for the 2nd and 4th samples:

Yadif(Mode=1,Order=1)
SRestore()#or SRestore(FRate=24.975) or RePAL

Man, I wish I could stop bugging you but I'm still confused about the bobbing part.

yadif is a deinterlacer correct? and you said I'm supposed to bob the video before using SRestore.
So in that example script wouldn't the output of SRestore() be something like 11fps because its lacking a bob?

EDIT : I just kicked myself in the ass for asking that question. I just read some of the yadif documentation (like I should have before bugging you) and Mode=1 is a bob.

1) Why FRate 24.975? I thought I was trying to get back the original 25fps.

2) You helped me before on another music video and it was the same script for the 2nd and 4th example except you told me to use SRestore(FRate=24). I was able to do what you stated (bob the video, take 50 fields and count how many unique, non-duplicate, non-field blended fields there were to determine it was 24fps source.) Is that the same process you used to figure out what to do with these sources?

3) How did you figure out what to do to example 3? (So that I can do it in the future and stop asking you all these questions.)

[manono]

1) Why FRate 24.975? I thought I was trying to get back the original 25fps.

The original was 25fps. And film is originally 24fps, yet on DVDs it's 23.976fps so that after the pulldown is applied it's output as 59.94 fields per second, as required. How does that slowdown happen? The drop_frame flag. For the same reason 25fps becomes 24.975fps on NTSC DVDs. Now, that's not always true, but the vast majority of the time it is. You'd have to count thousands of frames to be sure which is correct. Me, I usually just use RePAL (on blended sources) and forget about it unless I see too much remaining blending or missing frames from time to time. Do it however you like.

Is that the same process you used to figure out what to do with these sources?

Didn't I explain this already? First separate the fields and look for blending. If blended then figure out the pattern which leads you to the correct framerate and the unblending filter to use and its settings. If not blended after bobbing or separating the fields, start counting to figure out the pattern of field repeats which leads you to the correct framerate and how to achieve it. Like your sample 3; when you separate the fields or bob it, instead of the 32 32 32 pattern indicating telecined film, you get 32 322 32 322 indicative of a transfer from PAL. By the way, even though that's not a bad way to do the PAL2NTSC conversion and even though it's a helluva lot better than field blending, that's only the 3rd time I've ever seen it, after examining literally thousands of DVDs.

I was able to do what you stated (bob the video, take 50 fields and count how many unique, non-duplicate, non-field blended fields there were to determine it was 24fps source.)

I didn't have to do that this time as it was easy to figure the correct framerate. Getting there (for your first sample anyway) was the hard part.

[alcOre]

Didn't I explain this already?

Yes and no. I just had some questions as it related to 29.976i as opposed to 25i but I've got it figured out now with your help and some experimenting. I was able to go through most of the rest of my collection of vobs and process them correctly. Thanks again for your help.

I said most because I have one other source that I don't know what to do to it...its an animated music video and I thought it was progressive but after converting it and watching it, it also jitters. I'll post a clip later on.

[AlanHK]

Anyways, I have some more sources that I'm not sure what to do with. I also have a few questions. One, I've tried using SRestore on other sources that were similar to my other source (the music video that someone told me to use SRestore(FRate=24) because it was converted from a 24 fps source.) I took another source that I counted the unique frames and it was 24, just like my other source. I tried the same script on that one, and it came out jittery. I don't remember which source it was so I can't post it. I guess my question is when/how is one supposed to use SRestore?

I'll just comment on this part.

I've used SRestore a little, it's quite intriguing.
What I do is use it first as a diagnostic, just run it like:

AviSource("whatever.avi")
SRestore()

Then open that in VDub and see what the result is -- especially what SRestore chose as the frame rate.

For example, recently I had a cartoon at 29.997 fps, with lots of duplicate or blurred frames.
I was trying to IVTC it with poor results.

SRestore gave me an output of 12.5 fps, which I hadn't expected.

So I then made it

AviSource("whatever.avi")
SRestore(frate=12.5)
Changefps(25)

And made it PAL (since my target was DVD video, and both PAL or NTSC work for me).

I found it gave slightly better output, fewer blurred frames, if I used Changefps rather than
SRestore(frate=25)

So don't assume a frame rate, let SRestore guide you.

[manono]

Then open that in VDub and see what the result is -- especially what SRestore chose as the frame rate.

Hehe, you need a little bit more practice with it, or maybe a good reading of the included srestore.txt. It's usually meant to be preceded by a bobber. Then, if you go, for example:

Yadif(Order=1,Mode=1)
Srestore()

If the source DVD is NTSC, you'll get 25fps. If the source DVD is PAL, you'll get 23.976fps. Without a bobber preceding it you'll get half those framerates, which explains your 12.5fps result. It doesn't really "choose" anything for the framerate; those are just the default results. Use the FRate parameter to choose different output framerates.

I found it gave slightly better output, fewer blurred frames, if I used Changefps rather than
SRestore(frate=25)

You would have probably gotten even better results had you put on a bobber. There are sources, though, where it doesn't work worth a damn.

[alcOre]

Nevermind about that other source that I didn't know what to do with it (the animated music video.)

I just loaded my analysis script in vdub (source, AssumeTFF#Or AssumeBFF,SeparateFields()) and it looks exactly like my 3rd sample post.

1) EDIT : Actually now I'm confused lol. Its not exactly like my 3rd sample. My 3rd sample has 5 unique frames, then a duplicate. This animated music video source I'm talking about toggles between just full progressive frames (no duplicates) at some parts, then it goes into a pattern of 6 unique frames, then a duplicate. What do I do with this source?

EDIT 2 : Christ. After looking more through this animated source I see all kinds of patterns in it. Mostly I see just plain old
progressive encoding (AA BB CC, etc.) But sometimes I see the 5 unique + 1 dup pattern. I then saw at some
points a 4 unique + 1 dup pattern. And at one or two points I saw a break in the progressive field pattern. What I
mean by that is sometimes I saw a pattern like this ; A A B B C C D E F F G G H H. So now I really don't know what
the hell to do to this source lol.

2) Also, you stated this : Like your sample 3; when you separate the fields or bob it, instead of the 32 32 32 pattern indicating telecined film, you get 32 322 32 322 indicative of a transfer from PAL.

I don't see a 32 322 32 322 pattern in my 3rd example. Unless I'm confused on how you got that pattern.
What I'm seeing on the 3rd example (and my animated one) is 5 different progressive frames, and then a duplicate of the 5th frame following it.

So A B C D E E F G H I J J, or as fields, AA BB CC DD EE EE FF GG HH II JJ JJ

How does that pattern equal the pattern you mentioned?

[edDV]

1) Thanks for the reply manano...yah I know what the 3:2 pulldown is. But what i'm confused about is how the new progressive tvs display a progressive source picture. Like a bluray to a 1080p plasma. What's going on there? isnt the 24pfps -> 23.976p -> 29.97i (through 3:2 pulldown) for displaying an interlaced picture? What happens with progressive sources and tvs in order to display a progressive picture? EDIT : Nevermind. Read your posted article. Explained what I wanted to know about progressive to progressive tvs. I still would like to know how computer displays handle different things but hey, that's one question out of the way.

Depends on the standards region and the HDTV technology (e.g. native 50/59.94MHz or 100/120MHz).

Back to basics.

People need to understand how a DVD player and HDTV works before they get creative with encoding. I'll summarize, ask more as you wish.

1. "NTSC" 59.94Hz progressive TV (can be any panel resolution from SD to ED to HD). I'm ignoring scaling here.

480i/1080i 29.97 fps (59.94 fields per sec) input is deinterlaced or inverse telecined+frame repeated to 59.94p. Display refresh is always 59.94Hz. Quality of deinterlace and inverse telecine varies widely.

480p/720p 59.94 fps input is displayed directly. Progressive DVD players output 480p 59.94fps.

480p/720p/1080p 23.976 fps input (where supported) is frame repeated 2 then 3 then 2 etc. to 59.94 fps.

With few exceptions, any other frame rate must be presented as one of the above.


2 "PAL" 50Hz progressive TV (can be any panel resolution from SD to ED to HD).

576i/1080i 25 fps (50 fields per sec) input is deinterlaced or field weaved (film). Display refresh is always 50Hz. Quality of deinterlace varies widely.

576p/720p 50 fps input is displayed directly. Progressive DVD players output 480p or 576p at 50fps.

576p/720p/1080p 24 fps input (where supported) is sped up to 25fps and 2x frame repeated to 50fps or just frame repeated 2x to 48fps or 3x to 72fps (rare).

With few exceptions, any other frame rate must be presented as one of the above.


3 "NTSC" 120(119.88)Hz progressive TV (can be any panel resolution from SD to ED to HD).

480i/1080i 29.97fps (59.94 fields per sec) input is deinterlaced to 59.94p and 2x frame repeated or frame interpolated to 119.88 fps or if detected as film, inverse telecined to 23.976 fps and frame repeated 5x or frame interpolated to 119.88 fps. Display refresh is always 119.88Hz. Quality of deinterlace and inverse telecine varies.

480p/720p 59.94 fps input is 2x frame repeated or frame interpolated to 119.88 fps. Progressive DVD players output 480p 59.94 fps.

480p/720p/1080p 23.976 fps input (where supported) is frame repeated 5x or frame interpolated to 119.88 fps.

With few exceptions, any other frame rate must be presented as one of the above.


4 "PAL" 100Hz progressive TV (can be any panel resolution from SD to ED to HD).

576i/1080i 25 fps (50 fields per sec) input is deinterlaced or field weaved (film) and frame repeated 2x or frame interpolated to 100 fps. Display refresh is always 100Hz. Quality of deinterlace varies widely.

576p/720p 50 fps input is frame repeated 2x or frame interpolated to 100 fps . Progressive DVD players output 480p or 576p at 50fps.

576p/720p/1080p 24 fps input (where supported) is sped up to 25fps and 4x frame repeated or frame interpolated to 100fps or just frame repeated 4x to 96fps.

Bottom line, unlike multi-sync computer monitors, progressive TV sets are restricted to two or three input frame rates (per TV standard). You can't force a TV to accept other frame rates but you can fool it to see your custom standard as one of the three.

Issues may vary for the PC/Game VGA port.

[manono]

I don't see a 32 322 32 322 pattern in my 3rd example. Unless I'm confused on how you got that pattern.
What I'm seeing on the 3rd example (and my animated one) is 5 different progressive frames, and then a duplicate of the 5th frame following it.

So A B C D E E F G H I J J, or as fields, AA BB CC DD EE EE FF GG HH II JJ JJ

You're right, and I apologize. After going over it and reporting the results and how to handle it (correctly) in my first post, I forgot what I had seen when I got around to writing about it in that later post. It also has shifted fields, but separating the fields shows what's really going on. Even though it looks interlaced, it's really progressive, once the fields are lined up properly again. That is, after separating the fields you get (as you said):

AA BB CC DD EE EE

and after putting on TFM by itself, the interlacing disappears (fields are properly realigned) and you get this:

ABCDEE ABCDEE

That is, unlike telecined film where every 5th frame is a duplicate, for this PAL2NTSC conversion every 6th frame is a dupe, and you have to remove every 6th frame which you do as described above:

TFM()
TDecimate(Mode=0,Cycle=6,CycleR=1)

And animation can be difficult to figure out because it's supposed to have a ton of duplicate frames. It's often created at a framerate less than 29.97fps, and different parts often have different framerates. This is especially true of hand drawn animation where it's expensive to draw every frame for even 23.976 (24)fps. So you often get parts at 16, 12, or 8fps. Unless you actually have some 29.97fps parts, though, and unless it's blended, NTSC stuff is usually supposed to be 23.976fps.

[alcOre]

thanks for your reply manano...I can't tell you how much your knowledge is both greatly appreciated, and very helpful.

Honestly, that's what I figured had happened, because you are such an busy man replying to quite a few threads. And yah I read the documentation on tivtc to figure out what tfm did and tdecimate. I now have a greater knowledge of what the hell is going on, but I have much yet to learn.

I'm currently trying to figure out what the heck a script would look like to use mcbob and nnedi for bobbing, since it seems to be the best bobber out there that doesn't change the original frames (ala tempgaussmc.) I've gathered that once properly installed mcbob() should do the trick, but I don't know how to use it in conjunction with nnedi (which i believe will give better results than just mcbob alone, correct?)

I'm also looking at AnimeIVTC and reading the documentation to help me understand even better what's happening in my sources. Maybe AnimeIVTC would be a good fit for my animated music vids.

[alcOre]

when you separate the fields or bob it, instead of the 32 32 32 pattern indicating telecined film, you get 32 322 32 322 indicative of a transfer from PAL.

Funny that you said this...even though you were thinking of someone else's source and not mine, I just found one of my music videos that shows this 32 322 32 322 pattern. What filter should I use to fix this and what does the script look like? I think FDecimate is what I'm supposed to use but 1) I'm not sure if that's the best/highest quality one to use and 2) I don't know how to script it.

[manono]

Sample please.

If it's not blended (?), then no unblending is needed. Try TFM() by itself. If you get one dupe in 6 frames, like that earlier sample, use the same script:

TFM()
TDecimate(Mode=0,Cycle=6,CycleR=1)

[alcOre]

Yup no blending...just a straight 3 2 3 2 2 (fields...as in AAA BB CCC DD EE FFF GG HHH II JJ)

Here's the sample : http://www.mediafire.com/file/r2mrtzmmexo/KylieMinogue-Wow-Sample.m2v

[manono]

TFM()
TDecimate(Mode=0,Cycle=6,CycleR=1)