VideoHelp Forum

Try DVDFab and download streaming video, copy, convert or make Blu-rays,DVDs! Download free trial !
+ Reply to Thread
Results 1 to 24 of 24
Thread
  1. Member
    Join Date
    Apr 2001
    Location
    United States
    Search Comp PM
    I've been using the same script to prepare all of my 720p HD recordings from ABC and Fox for DVD, but lately I've read that ABC and come other networks are using a non-standard pulldown cadence.

    Here is the script I've been using so far:

    #LoadPlugin lines not needed if you copy Dlls to Avisynth plugins #folder
    LoadPlugin("c:\Program Files\AVISynth 2.5\plugins\DGDecode.dll")
    LoadPlugin("C:\Program Files\AVISynth 2.5\plugins\Decomb521.dll")
    MPEG2Source("F:\Video\RawCaptures\Terminator--The Sarah Connor Chronicles (Series Premiere_Pt2_Gnothi Seauton).d2v")
    ConvertToYv12()# Not needed, but Certain Encoders Want it
    Decimate(2)
    Decimate(5)
    BicubicResize(720,480)
    Sharpen(0.6)

    This particular script was used for the second episode of the "Sarah Connor Chronicles," but it's become my default script for 720p.

    What I'm wondering is if there's a command or plugin you can use with AVIScript that replaces the Decimate command that will look through the video stream and yank out the duplicates, regardless of the pattern they occur in. What I want to end up with is a purely progressive stream without any duplicates.
    Quote Quote  
  2. Hi-
    What I'm wondering is if there's a command or plugin you can use with AVIScript that replaces the Decimate command that will look through the video stream and yank out the duplicates
    No. You have to figure out the pattern and set up TDecimate accordingly. TIVTC's TDecimate is much more configurable for unusual framerates than is Decomb's Decimate.

    With any luck, though, the pattern will remain the same for that particular show, and maybe for several or many of the shows, so you shouldn't have to do the drudge work too many times.
    Quote Quote  
  3. Member
    Join Date
    Apr 2001
    Location
    United States
    Search Comp PM
    Thanks for the help.

    I added TDecimate back into the mix, and it seems to be working just fine. I did a test encode with one-minute clips from a preview for WTTW Chicago's Maximum Theatre (IMAX movies shown in HD) and Saturday's broadcast of "The Termainal" on ABC. I'll post the results later on tomorrow, but they're completely smooth.

    I'm going to use the script on tonight's episode of "The Sarah Connor Chronicles," but I'll have to wait before trying it out on 1080i material, since "ER" is on hiatus, and there's no other NBC shows that I'm even semi-interested in since they axed "Bionic Woman" and "Heroes" is on hiatus too.
    Quote Quote  
  4. Hi-

    1080i might be a bit tougher because you'll get a lot of orphaned fields which can be hard to work with.

    I don't cap myself (although I've done a ton of work with silent films where you also have to find a repeating pattern), but I've seen some 720p stuff where they've pulled out frames in a pattern so they could shorten the show and fit in more adds (I think), and you get some frames with no duplicate afterwards. That is, you can't even use Decimate(2) (or SelectEven()) in the script or you'll delete unique frames. Study the TDecimate doc thoroughly. You may find one of 3 modes useful; Mode=2, Mode=7, or Mode=1 defining the pattern. Once you find a repeating pattern, Mode=1 using Cycle and CycleR is probably the way to go. But maybe you've figured that out already.
    Quote Quote  
  5. I use SelectEvery() to restore 720p to 23.976 fps. You have to be careful and watch for pattern breaks though.

    I generally don't trust the automated decimation functions and use them only for the worst cases (where there are many pattern breaks). They often make mistakes and leave jerks in the video.
    Quote Quote  
  6. Member
    Join Date
    Apr 2001
    Location
    United States
    Search Comp PM
    Originally Posted by manono
    Hi-

    1080i might be a bit tougher because you'll get a lot of orphaned fields which can be hard to work with.

    I don't cap myself (although I've done a ton of work with silent films where you also have to find a repeating pattern), but I've seen some 720p stuff where they've pulled out frames in a pattern so they could shorten the show and fit in more adds (I think), and you get some frames with no duplicate afterwards. That is, you can't even use Decimate(2) (or SelectEven()) in the script or you'll delete unique frames. Study the TDecimate doc thoroughly. You may find one of 3 modes useful; Mode=2, Mode=7, or Mode=1 defining the pattern. Once you find a repeating pattern, Mode=1 using Cycle and CycleR is probably the way to go. But maybe you've figured that out already.
    That was exactly the problem I noticed with "The Terminal." One frame would repeat twice, the second one three times, the third one only once. It was all screwed up. Right now I'm using Mode=1, and yeah, that seems to be working right now.
    Quote Quote  
  7. Member
    Join Date
    Apr 2001
    Location
    United States
    Search Comp PM
    Well, I solved my problems and went back to using SelectEven() and TDecimate for 720p and TIVTC for 1080i.

    As I promised a few days ago, I finally got a chance to post the end results of my tests:

    http://www.megaupload.com/?d=BHLVHTL8 (Max)
    http://www.megaupload.com/?d=UWUS1VCV (Reaper)

    The first clip came from a 720p source, specifically a preview of WTTW Chicago's Maximum Theatre (IMAX movies in HD), and the second one came from a recent episode of "Reaper" in 1080i.
    Quote Quote  
  8. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    CubDukat,

    Gee. I was hoping you would stick to the "Lost" project. I wanted to see how *other* broadcasted
    areas were transmitting the picture, quality 'wise. As it turns out on my end, here in New York, (I get
    Conneticut stations) it seems that ABC aired pretty good quality for "Lost" yesterday and today. At
    least from the broadcasting station I am receiving from. You'rs (and others) mileage will prob vary.
    But I was curious non-the-less.

    With "Lost" you have to find the TEC keys. These are "frequecies" (an aid or part of the re-telecining
    that the broadcasters apply to these video contents these days) and there are at least -two- keys
    that you have to find and lock onto (in your decoding tool.. ie, AVIsynth function or algo) and then
    decode the telecine back to 24p. You should be able to do this in an AVIsynth script using the built-in
    SelectEvery() function, but you have to first find out what yours is in order to apply the above functions
    param values to. But watch out for the 'edits' they might have included in the source, cause your
    results might not work flawlessly if they <cut> anything, usually during scene-changes. I guess they
    are trying to squeez every minute into the hour (time slot) available, c/o the sponcers -- c$mmerc$als.

    Working this out with 29.970 fps sources is usually a little simplerr. But with some HD sources like
    those that are 59.94 fps, you have the extra frames to deal with -- the dups. And these confuse
    the process (the decoding or desifering aspects) when attempting manually. But, I'm in the middle
    of developing a tool (for advanced purposes) that aids in the manual part of all this nonsenes of
    TEC (Time Expansion/Compression) of video content as performed by the studio broadcasters these
    days.

    There are even worse TEC type sources out there, and not even HD. Check out the movie "The Matrix"
    for example. This one is a killer. But we all love a challenge. And that was one reason why and how
    I got the idea of building a tool for this. I don't know if I will ever complete it (to decode all the
    video contents under TEC) but I am giving it a my personal attention.

    Let us know if you did "Lost", cause I'd like to know how you made out. Thanks..

    -vhelp 4510
    Quote Quote  
  9. Let us know if you did "Lost", cause I'd like to know how you made out. Thanks..
    Yeah, it looks like in his last post he was dealing with shows where frames hadn't been removed, unlike the shows he was discussing earlier.
    you have to find and lock onto (in your decoding tool.. ie, AVIsynth function or algo) and then
    decode the telecine back to 24p.
    That's just it; the final result of these things isn't 23.976fps. It's faster than that, which allows the shows to complete earlier and allows more time for advertisements. The ones I've seen were around 25-26fps. And SelectEvery won't do the job because these patterns are often quite large (over 100 frames before repeating), and the cadence usually breaks from time to time. If you have a sample of this Lost TV show you'd like to upload, I can figure it out for you and explain how to do it.

    As for coming up with a program to figure it out and remove the dupe frames automatically, good luck, but I don't think it's going to happen. And anyway, there are already a couple of tools out there to handle at least part of the job (although I don't use them). neuron2's (Donald Graft's) Multidecimate filter for AviSynth runs a fast first pass and creates a text file of the metrics of the frames - the amount of change between the current frame and the previous frame. The ones with low or 0 metric are the dupe frames. You can use that to find a pattern for decimating those dupes, set the removal parameters and run a second pass to get rid of them. tritical's TIVTC can do something similar.

    If you'd like some help in figuring out a pattern for Lost or any other show, just upload an unprocessed piece of the source, a piece with movement, and I and others interested in the subject can have a look. That may not happen, though, because, as I remember, you have slow upload speed.
    Quote Quote  
  10. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    Evening everyone.

    manono,

    That may not happen, though, because, as I remember, you have slow upload speed.
    You are correct.. I'm still on dial-up. Once in a while I can sneak a few large files from work because
    we are on faster connection. That's one reason why I carry a 1-gig stick with me

    I know about all the other tools that do a lot in terms of ivtc methods, (and I wouldn't knock them either)
    but I was speaking from a personal goal point of view. I hate this nonsense with time exp/comp 'ing
    the video content via re-telecining. But the tool I'm working on is more or less for advance usage,
    where you might want to anallize the frames visually and inspect for various kinds of properties like
    dups; blends; ect. For me, it has now become a challenge to defeat the tec nonsense, but instead
    of avisynth or virtualdub as the tools, it will be somethine entirely offline or commandline/external.

    (thanks for the tips about neuron2's and triticle's feature for analizing via text dump/export)

    Anyway. I did a little more research on the lost clips I capture from my PCTV Pro hd card, and I
    found I was in error with my first comments about the two keys I mentioned. There's more nonsense
    going on (as you noted in your last response) and its become an even more challenge. Now, I want
    to incorporate a way to save the analizing of the frames I work on, for when I come back to it, later.
    I want to do this, so when I work on various videos, I have a way to restore last work progress.

    Well, I still would like to see some clips of LOST (for the quality of HD ) I can download them from
    work.

    manono, I will look into uploading a short (untouched HD) clip of LOST, from work.
    I'll try to find a scene where they pan. I prefer the ones where the scene is circling around
    as they pan, because this provides a good representation of real test for smoothness!

    side note --- you know when they aired the Wednesday and Thirsday (repeat) shows, I did
    not get a hint of "blips" from OTA. But as soon as the (NEW) episode came on, all of the
    sudden, I started getting these intermetting "blips" every so often. Unless I'm imagining things,
    I think something is going on around here with these broadcasters and *new* episode airings.

    -vhelp 4511
    Quote Quote  
  11. Member
    Join Date
    Apr 2001
    Location
    United States
    Search Comp PM
    I should still have last week's premiere on my hard drive. I'll chop out Hugo's chase sequence and post it, since that seems to have a lot of fast motion.

    I haven't watched it all the way through, but it looks like it came out fairly smooth. The only problem I seem to be having is the signal getting glitched, and I'm not sure whether it's because I'm using one antenna split between two tuners without re-amplifying the signal or it's just Vista acting pissy. I'm leaning more towards Vista, myself. I really wish I hadn't upgraded to it. Fortunately I still have my old XP drive, and if it weren't for the fact that it's so damn flaky, I'd just Ghost it onto the new drive, but I'm afraid it'll die in mid-transfer.
    Quote Quote  
  12. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    Are you guys sure ABC is altering the normal 3 frame 2 frame sequence for 720p? Telecine has nothing to do with 720p.

    I'm seeing 3:2 consistancy from FOX but haven't looked at Lost. I'll try to cap some next time.
    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  13. Member Soopafresh's Avatar
    Join Date
    Jan 2004
    Location
    United States
    Search Comp PM
    What I noticed from this season is the hyper-sharpening of the film. Someone went a bit overboard in post.
    Quote Quote  
  14. Originally Posted by edDV
    Are you guys sure ABC is altering the normal 3 frame 2 frame sequence for 720p? Telecine has nothing to do with 720p.
    I'm not sure of anything until I see a sample. I have seen samples of other shows in the past that have removed frames. I can't remember now if any of them were Lost samples, but I think some were ABC shows.
    Quote Quote  
  15. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    Originally Posted by manono
    Originally Posted by edDV
    Are you guys sure ABC is altering the normal 3 frame 2 frame sequence for 720p? Telecine has nothing to do with 720p.
    I'm not sure of anything until I see a sample. I have seen samples of other shows in the past that have removed frames. I can't remember now if any of them were Lost samples, but I think some were ABC shows.
    I wonder if this is FCC legal? If true.
    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  16. Why would removing frames be illegal? You can cut entire scenes out of movies. Many prerecorded radio shows remove all the silence between words. It's the same thing.
    Quote Quote  
  17. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    Originally Posted by jagabo
    Why would removing frames be illegal? You can cut entire scenes out of movies. Many prerecorded radio shows remove all the silence between words. It's the same thing.
    I'm not sure. ATSC (720p) is a standard, Can that standard be alterered on the prime channel and for what purpose?
    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  18. They're still showing 720p. They are simply removing frames (or repeats) from the source to reduce the running time. For example, instead of a 3:2 repeat pattern they could use a 2:2 repeat pattern and show the movie in 20 percent less time. There would still be 60 frames per second in the broadcast and on the TV.
    Quote Quote  
  19. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    Evening everyone.

    (I don't make any claims that I'm a pro at this, but this is what I have come to understand
    based on my dealings with this latest problems, thanks to the broadcasters)


    edDV, this is all Time Expansion related. The cadense is not the normal 3:2 we come
    to love. With this source HD source medium and the frame rate 59.94 and adding in those
    extra DUP's etc, make things a bit more tricky to decode in the usual 3:2 ivtc practice.

    You see, in the HD source (for 59.94 or 60p rounded ) you have the following (cadense) pattern:

    --> PDPDD

    WHERE: P=Progressive, and D=DUP

    This is the HD of 3:2 cadense pattern. Instead of the 3:2 (PPPII, PPPII ... ) you have the
    (PDPDD, PDPDD, PDPDD ... ) pattern layout. The two are equivalent in terms of IVTC or
    restoring back to 24p, aka FILM.

    Thus, as long as you see this (above) pattern throughout your source, you have the makings
    of a clean 60p-HD-IVTC decode back to 24p.

    How I've uncovered this in my HD type sources was based on my time spent dealing with this
    latest trend while examining one of my source, the wonderful tv series, LOST, though I have a
    lot of other sources I am examinging with, too. It has become a challenge within my limited time,
    though i'm still debugging this in my ongoing (broader) cadense analysis work via programming
    tools I develop to aid in the debugging side of things.


    But, for those who don't know.. to decode this to 24p, you remove one DUP frame and follow it
    by the usual 3:2 reduction system -- there are many variations of performing this aspect.

    partial AVIsynth script..
    ..
    --> selectevery(4,0,3)
    --> selectevery(5, 0,1,2,3)
    or, selectevery(5, 1,2,3,4)
    ..

    (There are other methods to restore back to 24p, but this above is basic and does work if
    the HDTV source is consistant and you start at a proper place inside the cadense, usually
    at the first
    PDPDD, using the above script)

    A problem might occur if the (above) pattern is interuped by a 'key' (that's what I've been
    calling it, for lack of a better term) and this key's purpose is to modify the 'speed' of the
    video. Every so often (or groups) you have these sequences of 'keys' that help to maintain
    a certain regularity of timing (cadense) process. Then, there are the more tougher cadense
    mechanisms that make your heart race with anger. I've seen where they add 'blends' (see
    below) to this routine. You see a blend, you better run. And our favoriate show, LOST, has'em.

    But, with todays HD sources (that also includes analog) -- basically speaking -- they are now
    time expanding it by modifying the telecine (or cadense) and this either speeds up or slows
    down the video to the point that it fits within the time slot (or then add or reduce the amount
    of tv commercials) Weather this latest trend is to serve the sponsers (commercials) or whatever,
    doesn't matter. This is what they are doing, today, and onward !!

    So as I was saying..

    There are many 'cadense' scenarios in all these HD type sources. They are not the same for
    every content. Sure, you might find a few that will follow a similar pattern/layout. But worse,
    they even include those retched 'blends'. And you can find this in (yes) the tv series, LOST !!
    (that is why you can't get a good lock with a 60p-HD-IVTC algo back to 24p from this tv series,
    and many series are following this pattern)

    (tomorrow, I will try and upload the sample video of this same sequence so that those of you
    who are following with interest may analize for yourself your own interpretation of the sequence
    of events, below

    But, for those who want to jump ahead and make your own observations, the scene in question
    here is during the opening, where Hugo is being chased by the police, and then it pans into Jack,
    what looks to be him preparing a short breakfast (ie, orange juice) as he is watching the news
    and begining to realize that it's Hugo)


    Below, is an partial demonstartion of a cadense analysis output from my HDTV source of LOST. It
    consists of 58 frames in all. And this is my idea of what I am comming to realize what is TEC
    (time expansion/compression) all about, and then some..

    Code:
           1    1    1     1    1    1    1    1    1    1    1    1
           2    2    3     4    4    5    5    6    6    7    7    8
    00124: 4567890123456789012345678901234567890123456789012345678901
           PDPDDPDPDDPDPDDPDPDDPBPDBPBPDBPBPDPbPDbPbPDbPPDDPDPDDPDPDD ..
    WHERE: 00124 is the starting position I choose in the analysis of the sources cadense patterns

    And, here is my interpretation of an example of the above as termed for what I call, time slice 'keys'
    (or whatever its unknown terminology or language supposively) might be called..

    --> PDPDD PDPDD PDPDD PDPDD -> { PBPDB PBPDB -> PBPD -> PBPDB PBPDB -> PPDD } -> PDPDD PDPDD ..

    (in describing what is going on in this setup cadense analysis above: the source starts off with
    the usual 60p-HD-IVTC pattern of PDPDD, then it branches off into time slices (until I find something
    better or proper to refer this as) or keys to adjust the length or play time of the video at that
    juncture, and then at the tail end, it regroups back to the 60P-HD-IVTC pattern)


    ..and, broken down to its component parts.

    ivtc --- PDPDD PDPDD PDPDD PDPDD ->
    keys -- PBPDB PBPDB ->
    keys -- PBPD ->
    keys -- PBPDB PBPDB ->
    keys -- PPDD ->
    ivtc --- PDPDD PDPDD ..

    WHERE: P=Progressive, D=DUP, B=Blend

    But, sometimes this strange cadense setup has nothing to do with time expansion/compression.
    I"ve seen where in some program contents where the video is always PDPDD, but will follow a
    short 'blip' or two. And, the sequence of PDPDD can be very long, even stretching across many
    scene-changes. And, I have even seen this in the tv series, LOST. Go check out that scene
    where Hugo is being chased by the police. I'm theorizing that there is a mixture of 'cadense'
    treatment for Special.Film.sequences vs. just.film -- if you know what I mean. And, in LOST,
    you have a lot of them. Take for instance, all those flash-backs and flash-forwards. This might
    be those special occasions where they don't apply any special cadense treatment. So, look
    for signs like this and possibly others in your HDTV sources moving forward in this endeavor.

    (But, before one goes running like a crazy person that they can't restore something from HDTV
    back to 24p, they better have made sure that their HD tuner card was not acting up. This
    is usually evident as glitches in the playing while viewing the video as it is recording. You
    will sometimes see the MPEG breaking up into pixelated blocks or green stripes or audio will
    glitch in and out. This is a clear indication that your HDTV source and it's 60p will not go
    so well back to 24p)


    I'm just so angry at the braodcasters for moving in this direction with the majority of their
    program contents. And we (unfortunately) have to comprimise and live with those that we
    can not restore. So, in the end, we have a decision to make.. keep it at 60p or reprocess
    to another format.. i.e., DVD, and live the conversion, good, poor, or bad.

    -vhelp 4520
    Quote Quote  
  20. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    I'll review ABC tonight and see what they are doing.

    In the past frames were repeated as I said:

    23.976p - 1234
    59.940p - 1122233444

    Fox still does it this way.

    Update1:
    "SuperNanny" is upscaled 720x480p/59.94 to 1280x720p/59.94
    I can advance frame by frame at 59.94.

    Next is Cashmere Mafia which I expect to be film based.
    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  21. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    edDV, the problem is, they (all) are not consistant. If you run one, at least don't
    assume that whatever you conclude that it is the same for all the contents, because
    they are not. You might want to observe several in order to get an idea of what is
    actually going on with this cadense business.

    -vhelp 4521
    Quote Quote  
  22. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    I'll be getting "Lost" tomorrow.
    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  23. Member edDV's Avatar
    Join Date
    Mar 2004
    Location
    Northern California, USA
    Search Comp PM
    Originally Posted by vhelp
    edDV, the problem is, they (all) are not consistant. If you run one, at least don't
    assume that whatever you conclude that it is the same for all the contents, because
    they are not. You might want to observe several in order to get an idea of what is
    actually going on with this cadense business.

    -vhelp 4521
    I'm seeing that with "Cashmere Mafia"
    There are long scenes with constant 3:2 cadence like this



    But there are many places with odd frame repeat patterns. I think some is generated by "stretch to fit" type commands. This one was an accelerating 2322233323 pan to reveal the sign

    Recommends: Kiva.org - Loans that change lives.
    http://www.kiva.org/about
    Quote Quote  
  24. Member vhelp's Avatar
    Join Date
    Mar 2001
    Location
    New York
    Search Comp PM
    Unfortunately, after uploading the files to www.megaupload.com and then copy/paste'ing the
    links to a notepad .txt file onto my mem stick, for some reason the text is all garbage dispite
    the countless times I opened and read this file while at work. The file is chineese here and
    nothing is fixing it. I'm pretty pissed at all the work (for nothing) and so I'm going to bail out
    of this one cause I lost my energy. And I'm just gonna sit down and realax and enjoy Lost.

    -vhelp 4522
    Quote Quote  



Similar Threads