PF24 footage in Avidemux. Help?

[Dcourtwreck]

I have some footage from a Canon HF M400 camcorder that I recorded in the pf24 frame rate mode. It is in a 60i container and I need to perform pulldown (I edit with Sony Vegas, it has no built in detection for that frame rate).

When opening the mts file in avidemux it detects the frame rate as 60 frames a second, rather than fields, which is wrong. I am able to get around that problem by first converting to avi. It is a lossless conversion, I used mp4camtoavi.

Now when I open the file with avidemux I at least get the proper 29.97 frame rate detected.

When I use the decomb telecide filter by itself, it does as it should, the interlaced frames are combined into 2 duplicates, the frames that were already progressive appear untouched.

I need those duplicate frames removed and the frame rate dropped to 23.976, but when I add the decomb decimate filter it just smudges every frame and doesn't delete the duplicates.

The filters are in the proper order, first is decomb telecide, then beneath is decomb decimate. The cycle count is correct (5). What am I doing wrong? Is there a simpler way of getting this done?

[poisondeathray]

I can confirm it doesn't work in avidemux. Looks like some bug. It might work on another version (either older or newer beta). Interestingly, when you push the preview button in the filter section it looks like it's working correctly

It's not necessarily "simpler", but an alternative method is with avisynth

or ffmpeg can IVTC now with -vf pullup -r 24000/1001

[Dcourtwreck]

Thanks, now I know that it is avidemux, not me

Do you happen to know where I can find a good guide to do ivtc in avisynth or ffmpeg?

[poisondeathray]

There is a bit of a learning curve for avisynth (it's all script based) , but it's very powerful for video manipulations, filtering etc...There isn't any good central resource except maybe the avisynth mediawiki (it's not that great to learn from)
http://avisynth.nl/index.php/Main_Page#New_to_AviSynth_-_start_here

The script is actually very easy for what you want to do, but it can be confusing for people new to avisynth or not that familiar with computers or programming:

DirectShowSource("video.mts")
AssumeTFF()
TFM().TDecimate()

ffmpeg is command line utility, but that makes it easy to batch process, say all the clips in a folder with a batch command line or batch file. I can help you with that - what format do you want to import into vegas ? A lossless codec like ffv1 or huffyuv will be very large in filesize, and ffv1 is sluggish to edit (very compressed) at HD sizes. Some people go with high bitrate MPEG2 because it's easy to edit and very compatible in vegas . Cineform would be fantastic (excellent editing performance and quality), but not offered in ffmpeg (you could use avisynth /vdub/cineform however)

[Dcourtwreck]

I don't think I have the space for lossless 1080 footage. For my actual editing in Vegas, I will use the original mts as a proxy. I only have problems with them when I render, so I will bring in the proper ivtc clips as a last step. A high bitrate mpeg2 should suffice, but I'm open to trying cineform if I can find easy enough instructions for avisynth.

[poisondeathray]

For ffmpeg from the commandline:

for %a in ("*.mts") do ffmpeg -i "%a" -vf pullup -r 24000/1001 -vcodec mpeg2video -q:v 2 -acodec copy -f mpegts "%~na.new.m2ts"

The -q:v is the quantizer. The lower the value, the larger the filesize, the higher the quality . People typically use values around 2-6


or if you want to do it from a .bat file, just replace the "%" with "%%"

for %%a in ("*.mts") do ffmpeg -i "%%a" -vf pullup -r 24000/1001 -vcodec mpeg2video -q:v 2 -acodec copy -f mpegts "%%~na.new.m2ts"
pause

[poisondeathray]

For my actual editing in Vegas, I will use the original mts as a proxy. I only have problems with them when I render, so I will bring in the proper ivtc clips as a last step.

You would need to use an IVTC'ed proxy, because the framecount is higher for the original clips, or proxy of the original clips (you couldn't just swap in an IVTC clip to replace the non IVTC'ed clip)

Alternatively, you could edit as interlaced 29.97, then IVTC the exported file at the end, but you run the risk of cadence breaks unless you're very careful where you cut

[poisondeathray]

avisynth is pretty simple to IVTC once you get going

1) install avisynth

2) Download the TIVTC package and put tivtc.dll in the avisynth plugins folder . Most .dll's are autoloading if placed int the plugins folder , but some need to be loaded manually or with LoadCplugin()
http://web.missouri.edu/~kes25c/

3) Open a file in notepad , copy & paste the script below. You need to change the paths (replace PATH , with the c:\folder or whatever, and names to match your file. Unfortunately you will need 1 script per file. But there are avisynth batch scripters and utilities

Save it, rename the extension from .txt to .avs

DirectShowSource("PATH\video.mts")
AssumeTFF()
TFM().TDecimate()

Avisynth is a frame server . You can open that .avs with any program that accepts avs scripts. To other programs, that .avs text file "looks" like an uncompressed IVTCed video. e.g. open the .avs with vdub and encode with whatever (e.g. cineform AVI).

You can actually frameserve into vegas with avfs , or sometimes ffdshow avs2avi works . But I wouldn't use DirectShowSource as a source filter for direct frameserving into vegas, because non linear seeks can cause frames to go out of order - that makes editing impossible. (Linear encoding without temporal filtering is fine with directshowsource.)

[Dcourtwreck]

I tried ffmpeg first, got an error relating to the "%a."

I followed the steps for avisynth, but when trying to open the file in virtualdub I get "AVI import filter error: (unknown) ." I added the TIVTC.dll and got a windows pop up asking me to confirm the action, does that mean it registered successfully?

Sorry, I'm not too familiar with doing things without a gui.

___________________

In Vegas, I've had success in the past with replacing converted media with that of a different frame rate (resolutions, compression too), although the frame rate has changed, the clip duration is the same. Vegas would drop in the replacement at the same starting point as the original, things would still line up (it might slightly nudge on cuts, as vegas snaps to frames and it is a different frame rate). Vegas detects the change in media, and changes how it will handle interlacing for the clip during render (no more crazy blending or choppiness thanks to a proper ivtc).

[poisondeathray]

for ffmpeg put the first %a in quotes:

Also, are your files .mts or .m2ts ?

for "%a" in ("*.mts") do ffmpeg -i "%a" -vf pullup -r 24000/1001 -vcodec mpeg2video -q:v 2 -acodec copy -f mpegts "%~na.new.m2ts"

For avisynth, do you have haali media splitter, or lav splitter installed and configured ? For directshow decoders, ffdshow or lav . On windows7 you can use preferred filter tweaker to adjust

DirectShowSource() relies on your system configured directshow filters and splitters. By default in Win7, the Microsoft decoders are configured and don't work so well with avisynth

In Vegas, I've had success in the past with replacing converted media with that of a different frame rate (resolutions, compression too), although the frame rate has changed, the clip duration is the same. Vegas would drop in the replacement at the same starting point as the original, things would still line up (it might slightly nudge on cuts, as vegas snaps to frames and it is a different frame rate). Vegas detects the change in media, and changes how it will handle interlacing for the clip during render (no more crazy blending or choppiness thanks to a proper ivtc).

Interesting. What timeline/project settings ?

Compression won't matter eitherway (as it's decoded to uncompressed on the timeline)

[Dcourtwreck]

Files are mts.

Yes, quotes in place. FFmpeg gives me an error in red.

%a no such file or directory

As a test, I tried to replace the "%a" with the location of the folder where the file was to see if that helped, but I got the same error. What does the "%a" command do?

I do not have a splitter installed. I guess that explains why my avs won't load.

Interesting. What timeline/project settings ?

For Vegas, project settings are one of the bluray templates - HD 1080-24p (1920x1080, 23.976 fps). It's a mixed media project, most of the footage is true 24p.

[Dcourtwreck]

For avisynth, do you have haali media splitter, or lav splitter installed and configured ? For directshow decoders, ffdshow or lav . On windows7 you can use preferred filter tweaker to adjust

Installed Lav splitter and ffdshow. Used the tweaker tool to set ffdshow as the default decoders. Virtualdub will now open my avi file that was losslessly converted from the mts, but attempting to open the avs gives me the same error:

AVI import filter error: (unknown)

Thanks for trying to help me out with this, but maybe I am beyond help

[poisondeathray]

For ffmpeg, it's just a wildcard

Open the command prompt in the same directory, or navigate to that directory using dos commands like cd folder . ffmpeg.exe has to been in the system folder , or the folder in question, or you have to specify a full path in the commandline, or setup as an environment variable



ugh... not sure about avisynth . It probably has something to do with your setup. Double check that paths are correct, maybe post your full script



The potential problem with using hard telecined (24p in 60i) on a 24p timeline is sometimes the wrong frame will be dropped in the 3:2 , 5 frame cycle, and depending on where you make edits. Yes, the overall durations match, but your edits won't necessarily match once you swap the proxy out with the true 24p

[aedipuss]

could you post a sample? anything would work. i'll see if the old HV20 pf24 hdv tools can be made to work with the h264/mp4 that cam produces.

[Dcourtwreck]

Open the command prompt in the same directory, or navigate to that directory using dos commands like cd folder . ffmpeg.exe has to been in the system folder , or the folder in question, or you have to specify a full path in the commandline, or setup as an environment variable

I used the cd command to change to the directory of ffmpeg, I have one of the video files in question in the same folder as the ffmpeg binary. Attached a screenshot of the command prompt.

My avisynth avs looks like this:

DirectShowSource("C:\Users\Derek\Desktop\avchd_convert_v9\00002.mts")
AssumeTFF()
TFM().TDecimate()

You will notice I am linking to the video file inside a avcd_convert folder, I had tried using that automated batcher to generate my avs. The batcher was not used for this avs, the above code is the one I created after reading your post, I just used the video file I had placed in that folder.

could you post a sample? anything would work. i'll see if the old HV20 pf24 hdv tools can be made to work with the h264/mp4 that cam produces.

Sure, attached a sample. This clip is from a wedding. It isn't pretty, lighting was bad (for this little camera). There is plenty of movement though, to easily identify the interlaced frames. Luckily it was nice and bright in the center of the sanctuary.

[poisondeathray]

For ffmpeg you don't have the correct directories . ffmpeg is in the "...bin" folder, but presumably your video files are not in that directory . The command line should be called within the folder of files to be converted, and if ffmpeg isn't in the same directory, or not in the system path , that path has to be specified

replace "PATH\TO" to whereever you put ffmpeg.exe

for %a in ("*.mts") do "PATH\TO\ffmpeg.exe" -i "%a" -vf pullup -r 24000/1001 -vcodec mpeg2video -q:v 2 -acodec copy -f mpegts "%~na.new.m2ts"

EDIT: whoops, I missed that sorry:

I have one of the video files in question in the same folder as the ffmpeg binary.

But it looks like you are starting the commandline with "ffmpeg for..." , not "for..."


It might be easier to do it with a batch file

1) Place a copy of ffmpeg.exe in the directory of files to be converted

2) Open a text file in notepad in the directory of files to be converted, type the following, save it , change the extension from .txt to .bat, (e.g. "batch.bat")

for %%a in ("*.mts") do ffmpeg -i "%%a" -vf pullup -r 24000/1001 -vcodec mpeg2video -q:v 2 -acodec copy -f mpegts "%%~na.new.m2ts" 
pause

3) double click the .bat file



So did the avisynth method work for you ? Can you load the .avs in vdub ? post any error messages verbatim

[Dcourtwreck]

But it looks like you are starting the commandline with "ffmpeg for..." , not "for..."

That is where I messed up. I incorrectly assumed that I needed to have ffmpeg listed first. Before I saw your edit I just did the batch file, so ffmpeg is doing its thing now. It has output a couple of the smaller files, the frame rate is correct and decomb/decimate did well! These are some clips with poor lighting, I may run it again with the quantizer at 1.

So did the avisynth method work for you ? Can you load the .avs in vdub ? post any error messages verbatim

I still can't get avisynth working. I think the .avs is correct, but I must not have lav/ffdshow configured right. When using Win7DSFilterTweaker_5.9 I've tried setting both 32bit and 64bit to either lav or ffdshow for the decoder, but I still get the error. Although I don't plan on using avidemux to work with the avs, I know that it is capable of opening an avs, so I tried that as well. Attached an image with the error from both vdub and avidemux.

[poisondeathray]

Are you using 32bit everything ? Avisynth 64bit is unstable / has lots of issues. That means 32bit avisynth, 32bit vdub, etc....

Can you make a 1 line script

version()

save it, and open that .avs in vdub

[Dcourtwreck]

The one liner will not open either, same error. I didn't know the 64bit version was trouble, I'm going to have to go back and go 32bit everything. Will report back.

[poisondeathray]

64bit OS is ok, just all applications in the avisynth chain have to be 32bit, or have some wave of handling 32bit => 64bit issue

[poisondeathray]

khaver made some batch intermediate tools with GUI's back a few years ago . They have PDF instructions and screenshots. They were meant for HV20 (HDV), but should work with anything (avisynth based). I don't know if there are updated versions, but he posts at this forum as well.

http://hddv.net/showthread.php?35295-Batch-3-2-Pulldown-Remover
http://hddv.net/showthread.php?29578-Batch-Intermediate-Creator-New-Version

[aedipuss]

there is an old tool that will ivtc the 30i to an intermediate 24p that will edit in vegas pro fine.
search for - Batch_32_Pulldown_Remover_v14

install it and any of the other programs it needs if you don't already have them. install the UT codec. use the ut yuv420 bt709 compression.

use the output intermediate 24p video and the audio from your original 30i. the wav the program file creates is always to short.

[Dcourtwreck]

just all applications in the avisynth chain have to be 32bit, or have some wave of handling 32bit => 64bit issue

After going 32 bit, I can now open the test script in virtualdub, and the ones that link to my mts files. So that means I can create the files through virtualdub now if I choose to. Cool beans

khaver made some batch intermediate tools with GUI's back a few years ago . They have PDF instructions and screenshots. They were meant for HV20 (HDV), but should work with anything (avisynth based). I don't know if there are updated versions, but he posts at this forum as well.

there is an old tool that will ivtc the 30i to an intermediate 24p that will edit in vegas pro fine.
search for - Batch_32_Pulldown_Remover_v14

I did locate khaver's tools, looks like the newest is v16. Since avisynth is now working I can give it a try.

You can actually frameserve into vegas with avfs , or sometimes ffdshow avs2avi works . But I wouldn't use DirectShowSource as a source filter for direct frameserving into vegas, because non linear seeks can cause frames to go out of order - that makes editing impossible. (Linear encoding without temporal filtering is fine with directshowsource.)

Frameserving into vegas has my curiousity peaked. I could edit with the ivtc clips I have created, then replace with the frameserve for render. I wouldn't need to seek since the actual edit is complete.

Even if that doesn't work out, I do have proper ivtc clips now.

Thank you poisondeathray and aedipuss, I really do appreciate the help

I may end up having more questions, but I felt I needed to throw a "thanks" in.

[poisondeathray]

You can actually frameserve into vegas with avfs , or sometimes ffdshow avs2avi works . But I wouldn't use DirectShowSource as a source filter for direct frameserving into vegas, because non linear seeks can cause frames to go out of order - that makes editing impossible. (Linear encoding without temporal filtering is fine with directshowsource.)

Frameserving into vegas has my curiousity peaked. I could edit with the ivtc clips I have created, then replace with the frameserve for render. I wouldn't need to seek since the actual edit is complete.

Even if that doesn't work out, I do have proper ivtc clips now.

What would be the reason for that? For a "lossless" workflow ? Concerned about the loss incurred with MPEG2 ?

It's still "dicey" to do it with non indexed clips such as doing it with DirectShowSource()

Loading AVCHD clips with avisynth can be problematic, especially when you need frame accuracy and temporal seeking

If you get more involved with avisynth , you might consider getting DGNVTools if you have an Nvidia card, it's more reliable but not free

DSS2 is supposed to be frame accurate, but relies on container timestamps and has a last frame bug (it always missing last frame, and sometimes "hangs" when encoding to the end)

FFMS2 usually has frame repeats and issues with frame rate

A reportedly way that works with AVCHD is L-Smash Source, using LWLibavVideoSource() with repeat=true, but there are still some people that have problems getting it work properly with AVCHD for whatever reason

Also, to frameserve into vegas, you must convert to RGB. And most "lossless" intermediates such as lagarith, huffyuv, ut video codec, are not treated as YUV by vegas, they are treated as RGB . That has big implications when you need to salvage superdarks or superbrights, as they might be clipped depending on how you have vegas setup (most non-native camera formats are converted using "computer RGB" , not "studio RGB") . So for a dark footage it can be dangerous unless you either correct for it beforehand, or use an intermediate that is treated as YUV

[Dcourtwreck]

What would be the reason for that? For a "lossless" workflow ? Concerned about the loss incurred with MPEG2 ?

Yes, that was the goal. But after reading the rest of your post it seems that it isn't worth the trouble of trying to frameserve, since it has to be converted to RBG first. If I had to create a lossless intermediate I might as well just bring them in directly, rather than frameserve in (assuming I understood that correctly).

If you get more involved with avisynth , you might consider getting DGNVTools if you have an Nvidia card, it's more reliable but not free

Interesting, but I have an amd card. So that it out.