VideoHelp Forum
+ Reply to Thread
Results 1 to 17 of 17
Thread
  1. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    2 days into this but still stumped. 1925 silent movie recorded off cable tv to DVD-R. Pulldown is easy to see, but after IVTC and various settings I get uneven patterns of dupe frames. One pattern looks like AABBCDEEFGHHIJJKLMNN, etc. Used various combos of TIVTC and even sRestore to try for 18 to 20 fps, but get dupes mixed with triplicate and missing frames which looks worse.

    Hunted down some earlier threads including this oldie: http://forum.videohelp.com/threads/312297-Adjusting-frame-rate-on-a-slient-film and similar threads. But no winner from any method I could find, and I'm running out of mode and cycle number combos. Attached samples are unprocessed cuts from VOB using DGIndex. Jerky motion is more obvious on sample2.
    Attached Files
    -ann's brother
    Quote Quote  
  2. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Another way to approach this is to use multidecimate since it's a silent film. Or dedup, which outputs VFR timecodes which can keep everything in sync if there was audio

    I used dedup here. Basically you run the 1st analysis pass, then use the debugmode (show=true) to see what thresholds are required for your content . For example the 2nd clip has some "noisy" duplicates (from compression, whatever) that might not be detected as "duplicates" if you didn't adjust the threshold. Once you walk through the process it's pretty easy.

    You can encode with VFR timecodes as I did here, or if you want, you can do a CFR encode by looking at the original clip time and look at the decimated framecount to figure out the FPS



    EDIT: oops uploaded wrong video for 2nd one.
    Attached Files
    • File Type: mp4 1.mp4 (2.40 MB, 8 views)
    • File Type: mp4 2.mp4 (5.67 MB, 5 views)
    Last edited by poisondeathray; 13th Jul 2014 at 14:37.
    Quote Quote  
  3. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Yep, the mp4 is pretty close to what I got using other methods. It does get a little weird at 18fps. Like, frames 51, 64, and 77 repeat 3 times and there are still dupes. At least nothing got dropped in the mp4. Now, unless this movie was shot at 10 fps I started thinking there's something wacky about the print that was broadcast. I figured if I got it up to 18 or 20fps I could flag it for pulldown with DGPulldown. But just too many oddities with this particular copy of that movie.

    Looks like I have some learning to do with DeDup and Multidecimate. Thanks for the ideas. I'll give 'em a whirl.
    -ann's brother
    Quote Quote  
  4. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Yes, IMO it's more important to err on the side of conservatism to retain duplicates, than to decimate a "good" frame

    When you look at the debugmode and threshold values, you will see there is a fairly large buffer range - you have a fair bit of room. The value that works for both clips in this case is 4.2. The only reason it's that "high" is because the 2nd clip has "dirty" duplicates that have compression noise. So if these are representative samples, that buffer suggests it's a good chance that value will be "perfect" for the entire film. This might be going over your head right now, but I assure you once you run through it a couple times, it's very straight forward to use dedup

    The 1st clip is close to 18 FPS (if you average out the timecodes), but the 2nd clip is closer to 16 FPS . The fact is, silent films of that time were often variable frame rate, even between scenes. And if the desired output is DVD-video, then VFR won't work for you .
    Quote Quote  
  5. I see the first sample as 18fps and the second as 16fps. Maybe it was done purposely to make it look like the march through the forest was more ominous and dreadful. Wonderful film, The Big Parade.

    I won't claim these are perfect but I don't believe any frames are dropped:

    First sample:

    TFM()
    Tdecimate(Mode=0,Cycle=60,CycleR=24)


    SecondSample:

    TFM()
    Tdecimate(Mode=0,Cycle=120,CycleR=56)


    When I worked with it to make a DVD, I believe (if memory serves) I made the whole thing 18fps and then frame-interpolated up to 19.98fps so I could run DGPulldown for 3:3 pulldown, the lowest framerate for this to work with NTSC DVDs.
    Last edited by manono; 13th Jul 2014 at 17:20.
    Quote Quote  
  6. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by LMotlow View Post
    Yep, the mp4 is pretty close to what I got using other methods. It does get a little weird at 18fps. Like, frames 51, 64, and 77 repeat 3 times and there are still dupes.
    Are you sure about that ? Which mp4 are you referring to ?

    Are you using a frame accurate preview method ? Your decoder might be adding duplicates

    As far as I can tell there are no duplicates, and no missing frames



    In fact you can double check with dedups' debugmode by using dec=false, show=true . Dec=false means no decimation is done, so you just see the overlay information
    Last edited by poisondeathray; 13th Jul 2014 at 15:53.
    Quote Quote  
  7. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Originally Posted by poisondeathray View Post
    Originally Posted by LMotlow View Post
    Yep, the mp4 is pretty close to what I got using other methods. It does get a little weird at 18fps. Like, frames 51, 64, and 77 repeat 3 times and there are still dupes.
    Are you sure about that ? Which mp4 are you referring to ?

    Are you using a frame accurate preview method ? Your decoder might be adding duplicates

    As far as I can tell there are no duplicates, and no missing frames
    It opens as you describe it only with DirectShowSource. No dupes, nothing missing. Runs in Avisynth/VitualDub at 25fps. I used assumefps(18.00) as a quickie fix and it still looked smooth. If I open in NLE's, media players, VirtualDub directly, etc., I get dupes.

    @poisondeathray & manono: I'll try it with all three methods, IVTC, Multidecimate, and DeDup. May as well learn this because relatives are coming at me with with this kind of oddball stuff and they want me to fix it. Right, I better start charging these folks or at least get some good pancake breakfasts out of it, LOL! I have it on DVD, and tell the truth I haven't even watched that DVD yet. I got the samples from something I recorded by accident. I saw something on tv and hit Record on the DVD-R, then got involved in chores and forgot it. 4 hours later, I remembered Record was still going, but I sat and watched the Big Parade broadcast and saw this clumsy motion, so I let the movie finish. Surprised that they broadcast it this way.

    If I recall when I saw this flick at the Museum of Modern Art a few years back, the attack scenes moved slower than I see here. Or maybe my memory ain't so great. Anyway I'll give all three methods a try. Want you to know I appreciate all this feedback and will be able to make use of it.
    -ann's brother
    Quote Quote  
  8. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    With VFR encodes, duplicates are inserted by the playback software to preserve the timing. They don't exist physically in the bitstream. If you check with indexed frame accurate source filter (e.g. ffms2, dgavcindex, l-smash), you will see it is a 1:1 copy of unique frames . IMO it's important to have a 1:1 copy if you ever need to do anything like clean it up, restore etc...

    It's tough to hand crank and keep a CFR. Even skilled operators are human.

    All methods of FPS determination used here are based on the assumption that the broadcast was correct in the first place. If that's not true (e.g. someone screwed up somewhere), then you can adjust it to what you think it should be
    Quote Quote  
  9. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Thanks again, poisondeathray. DGavcIndex is the only one I didn't try, although DirectShowSource at defaults gave me a smooth video.

    I realize the youngsters don't go for silent stuff, but damn when those people set out to make a good flick they sure knew what they were doing with that old hardware. I do notice a lot of silent scenes move at a different pace, and it looks like playing with crank speeds was often done on purpose. It's still done today, and usually overdone. If I see one more leap through glass or gasoline explosion in slo-mo I think I'll just give up and turn the movie off.
    -ann's brother
    Quote Quote  
  10. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Sorry, actually DGAVCIndex doesn't work on it - it's depreciated anyway we shouldn't be using it

    You can use ffmpegsource2 (aka ffms2) , and use seekmode=0 if you want it more accurate and are doing non linear seeks
    FFVideoSource("1.mp4",seekmode=0)

    Directshowsource isn't frame accurate, and relies on your installed directshow decoder. So it's unreliable and inconsistent between systems.

    If we want an equivalent CFR timing, just divide the framecount by the clip duration
    e.g.
    clip1 has 156frames / 8.5 sec =~ 18.3fps
    clip2 has 224frames / 14sec = 16fps
    Quote Quote  
  11. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Originally Posted by poisondeathray View Post
    You can use ffmpegsource2 (aka ffms2) , and use seekmode=0 if you want it more accurate and are doing non linear seeks
    FFVideoSource("1.mp4",seekmode=0)

    Directshowsource isn't frame accurate, and relies on your installed directshow decoder. So it's unreliable and inconsistent between systems.
    That's curious, because I've heard the opposite. Anything from ffms2 works only when the Moon is just right, and DirectShowSource is always wll behaved. Then someone else claims DirectShowSource works only when the groundhog doesn't see its shadow, and MPEG2Source is the only choice for VOBs. The source is unprocessed VOB, so I guess you have to try all three or find a fourth and fifth to compare. It's a long slow haul dealing with software you can't trust, and most of that time and toil is wasted.

    Anyway, DeDup hasn't De-duped so far. Maybe AviSource() isn't being frame accurate. But the bytes for pass 1 output and pass 2 output are exactly the same, so I guess the Moon is right for AviSource. I can also see how very static or very fast scenes could make DeDup go haywire. Maybe one of the other dedupers will be possible to navigate, and I guess I should browse doom9 to see what developers have to say about their dedupers.

    I'd say the original broadcast was full of bad beans. The recording is still on the DVD-R and plays with the same chunky motion as the VOB on my computer. Just called a friend in Nashville who recorded the same broadcast on his cable DVR and said it looked like trash. While fishing around some archives on hard drives I found some HD PVR recordings from that provider (TCM) that play the same way. Had problems with TCM on several occasions. Smart-rendering editors wouldn't pass them without full re-encodes. I should stick with retail DVD and BD for those. This recording of The Big Parade appears to be from tape anyway.

    Meanwhile this oddball capture stays on my main PC for a couple of weeks as something to learn from.
    -ann's brother
    Quote Quote  
  12. Member
    Join Date: Sep 2007
    Location: Canada
    Search Comp PM
    Originally Posted by LMotlow View Post
    Directshowsource isn't frame accurate, and relies on your installed directshow decoder. So it's unreliable and inconsistent between systems.
    That's curious, because I've heard the opposite. Anything from ffms2 works only when the Moon is just right, and DirectShowSource is always wll behaved. Then someone else claims DirectShowSource works only when the groundhog doesn't see its shadow, and MPEG2Source is the only choice for VOBs. The source is unprocessed VOB, so I guess you have to try all three or find a fourth and fifth to compare. It's a long slow haul dealing with software you can't trust, and most of that time and toil is wasted.

    Yes, without a doubt ,for a VOB or MPEG2 source, MPEG2Source() is the most consistent

    Yes, without a doubt, DirectShowSource() is the most inconsistent for ANY source, and not necessarily frame accurate . Basically different systems can have different directshow splitters, decoders, and filters in the pathway which will pollute your results. Even if you know exactly how your directshow chain is configured, it's still frame inaccurate with non linear seeks.

    Yes, FFMS2 has many "quircks" such as slightly "off" FPS in some containers (because it reads container timestamps), problems with interlaced AVC streams, but it is frame accurate because of the index . But for an AVC/MP4 source , it along with l-smash are the only reliable free options. (DGNVTools is another reliable option, but not free, and relies on a compatible Nvidia card)


    Anyway, DeDup hasn't De-duped so far. Maybe AviSource() isn't being frame accurate. But the bytes for pass 1 output and pass 2 output are exactly the same, so I guess the Moon is right for AviSource. I can also see how very static or very fast scenes could make DeDup go haywire. Maybe one of the other dedupers will be possible to navigate, and I guess I should browse doom9 to see what developers have to say about their dedupers.
    Very static scenes that are below the threshold will be decimated. But , by definition, a "Very" static scene can be represented by a single frame. It wont' go out of sync on content with audio, because of the timecodes . But infinitesimal motion, like a microscopic flutter of an eyelash will cause problems - there won't be enough room to distinguish between "duplicate" vs. non duplicate accurately

    Very fast scenes are less likely to be problematic, but there is a trigger2 and threshold2 value which are used

    The most common problem is noise. Adjacent duplicates with varying noise patterns on "top" will be detected as "different" and not decimated

    Like any "auto" detection filter, it can make mistakes, there are manual overrides for that "ovr" .



    More clear instructions:
    1) same as normal IVTC, but the 1st pass is just an analysis pass and writes the metrics to a log file. It's just measureing the difference between frames, detecting duplicates. You dont need to save to AVI. Just run vdub video=>run analysis pass. Uncomment out the 1st DupMC line when you run the 1st pass, comment out the actual Dedup line, save the avs


    Code:
    MPEG2Source("Sample1.d2v")
    AssumeTFF()
    TFM()
    TDecimate()
    DupMC(log="sample1log.txt")
    #DeDup(threshold=4.2, maxcopies=20, maxdrops=3, log="sample1log.txt", times="timecodes1.txt", dec=true, show=false)

    2) once the analysis is done, and log file written, change the avs to comment out the DupMC line, and uncomment out the DeDup line, save the avs

    Code:
    MPEG2Source("Sample1.d2v")
    AssumeTFF()
    TFM()
    TDecimate()
    #DupMC(log="sample1log.txt")
    DeDup(threshold=4.2, maxcopies=20, maxdrops=3, log="sample1log.txt", times="timecodes1.txt", dec=false, show=true)
    To debug it, use show=true and dec=false and step through looking at the overlay. There are other parameters to tweak, but this will get you started. The main parameter to adjust is the threshold. If you look through both clips, the value you need is actually 4.16. You want it to include only true duplicates, but not miss any. It's normally not that high for for "clean" clips - The default value is only 0.3. If you look at the 2nd clip, at default settings Tdecimate using a larger cycle and cycleR will mistake a duplicate for a non duplicate, because of the noise and shifty compression issues at decimated frame 140-141. But I suppose it might be possible to adjust the tdecimate settings if you wanted to use that. If you had more than strings of 3 dupes in the IVTCed input , you would have needed to increase the maxdrops. To actually decimate, use dec=true

    Of course you would use a different logfile and timecodes for another clip, as done in these 2 examples, but you should be running it on the entire film as 1 dedup run. There should be enough "headroom" on this content that you don't have to manually do it per scene



    I'd say the original broadcast was full of bad beans. The recording is still on the DVD-R and plays with the same chunky motion as the VOB on my computer. Just called a friend in Nashville who recorded the same broadcast on his cable DVR and said it looked like trash. While fishing around some archives on hard drives I found some HD PVR recordings from that provider (TCM) that play the same way. Had problems with TCM on several occasions. Smart-rendering editors wouldn't pass them without full re-encodes. I should stick with retail DVD and BD for those. This recording of The Big Parade appears to be from tape anyway.
    The decimated motion is smooth (there are no big drops or gaps in motion, no duplicates or stutters) , but any content that runs at a true rate of 16-18 FPS is going to look "chunky" no matter what you do. I don't know what the "actual" rate of the original, original source was supposed to be - but there doesn't look to be missing any frames at least on these 2 samples. I suspect that is the actual rate
    Last edited by poisondeathray; 13th Jul 2014 at 23:24.
    Quote Quote  
  13. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Yep, after got more accustomed to this plugin I managed to get good results. I ended up with threshold about 4.8 and maxcopies 20 in one scene, slightly different numbers for others. Likely I'd have to work it scene by scene like old VHS captures. I have another TCM recording thru an HD PVR (55 Days At Peking) with similar issues during the movie but not during the usual TCM intro and epilog. I'll give the other plugins a try during this week. Good thing this wasn't a once in a lifetime event that can't be improved by getting a commercial issue.

    Yes, at 18fps or so after feeding results thru DGPulldown, things don't move as smooth as modern film, but it sure was improved. I also made a little headway with manono's suggestions. Manono mentioned motion interpolation, so you two guys will keep me busy for days to come! Thanks to both of you for opening the door to something new, again.

    ED 7/14 -- Forgot to mention, I get similar problems in some HD PVR captures edited on the PC. Some of the ideas in this thread will come in handy, I think.
    Last edited by LMotlow; 14th Jul 2014 at 06:40.
    -ann's brother
    Quote Quote  
  14. Originally Posted by LMotlow View Post
    Yes, at 18fps or so after feeding results thru DGPulldown, things don't move as smooth as modern film, but it sure was improved.
    If you really ran it through DGPulldown for 18->29.97 (or 17.982->29.97), did you happen to notice that you changed the length in the process? Didn't you read what I wrote earlier:
    Originally Posted by manono View Post
    ...I made the whole thing 18fps and then frame-interpolated up to 19.98fps so I could run DGPulldown for 3:3 pulldown, the lowest framerate for this to work with NTSC DVDs.
    There are also tricks that will allow you to get better encoding results for DVD, ones such as adjusting the bitrates because you're not allowed to encode for 17.982fps and the sizes won't come out as you expect.
    Last edited by manono; 14th Jul 2014 at 16:04.
    Quote Quote  
  15. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Your method looks correct to me. My target is 19.98 fps, and get there one way or another when I work later with silent stuff that's below that speed. That run thru DGpulldown was just a quickie. Of course, it didn't work so great, LOL! Encoded file size isn't a priority with me.
    -ann's brother
    Quote Quote  
  16. Originally Posted by LMotlow View Post
    Encoded file size isn't a priority with me.
    But maybe encoding quality is?

    Say you set a max bitrate of 9000. You encode at 23.976fps for a file that's really eventually going to play at 19.98fps. 19.98/23.976=.8333...
    After DGPulldown your effective max bitrate will become about 7500. The same applies for the average bitrate. So, when you really want 19.98fps when all done, boost all bitrates by a factor of 1.2 as compared to what you would ordinarily use for the 23.976fps encode.
    Quote Quote  
  17. Member
    Join Date: May 2014
    Location: Tennessee, US
    Search Comp PM
    Understood, manono, and thanks for the extra insight on those numbers. Never thought of bitrate in that way, but it makes sense. My priority is the best PQ I can muster. Why work like a horse just to end up with chicken feed for results?
    -ann's brother
    Quote Quote