Going from VirtualDub on a PC to FCP X on a MAC

[a1s2d3f4]

I am trying to prepare some video files on a Windows machine for my friend who uses FCP X on his Mac.

I have always used AVISynth with VirtualDub and encoded my videos to XVID (maximum quality).
But this style of compression is not supported by FCP.

I am looking for two possible solutions:
1) Figure out how to output my videos from VirtualDub so that they look no worse than XVID and don't take up more disk space and are in a format that FCP accepts
or
2) Figure out some tool that he can use on his mac to losslessly convert my XVID videos (I can re-wrap them into MOV container using ffmpeg if necessary) to some FCP-friendly format.

Any help is highly appreciated.
a1

[sneaker]

Try using the "external encoders" feature of VirtualDub to create H.264+AAC(or maybe PCM?) in MP4 container.

[a1s2d3f4]

Never used it before, but thanks for recommending.
Looking at it. Ok, complicated.
Googled.
Found this page: https://encodingtalk.com/threads/virtualdub-save-direct-to-mp4-fraps-to-youtube-with-e...encoders.2446/
Read through it.
Clicked on ffmpeg: https://encodingtalk.com/threads/virtualdub-save-direct-to-mp4-fraps-to-youtube-with-e...e-2#post-13731
Copied that code, did all the stuff.
Then tried to export a test video to MOV using the so-called "V ff 264 r22".
Its parameters are:
-f rawvideo -s %(width)x%(height) -r %(fps) -i - -vcodec libx264 -pix_fmt yuv420p -crf 22 -preset medium -x264opts colormatrix=bt709:vbv-maxrate=62500:vbv-bufsize=78125 "%(tempvideofile)"

When I exported this test video to XVID, I would get a good looking file at 883MB (960x540 59.94fps ca 10mins).
Using this ffmpeg conversion, it reported that it will create a huge file - 27GB (but now, I think it was thinking of it as a "lossless" export within VDub).
I stopped it after a minute and then discovered that the file created was actually very small in size, but it was awful to look at. (Low quality).
So, perhaps, if I knew how to adjust the parameters to increase the file size and increase quality, it would be possible to achieve what I want.

Any ideas on how to change ffmpeg parameters, or should I ask in a different thread?

Clearly something is not quite right here.

[sneaker]

Decrease the -crf value to increase quality (-3 ~= double size). Ignore the "27 GB" thing, it is indeed just the size of the lossless data being piped from VirtualDub to the encoder, not the size of the resulting file on the HDD.

[a1s2d3f4]

cool, will do. What about vbv-maxrate/bufsize variables? Should I also do things there?

Also, a lot of my videos are B&W if it matter at all.

[sneaker]

cool, will do. What about vbv-maxrate/bufsize variables? Should I also do things there?

They seem ok. (vbv is usually just important for playback, doesn't matter much for editing)

Also, a lot of my videos are B&W if it matter at all.

I would add Grayscale() to the AviSynth script to make sure the chroma panes are all 0. Then it will compress extremely well. Nothing special needs to be added to the command-line otherwise.

[a1s2d3f4]

Well, tried crf15 - still losing quality, but definitely much better.
I'll try 7, or something.

What worries me slightly more is that after conversion I have "10 frames" of nothing added to the front of the output video. and the audio has become slightly out of sync (not by 10 frames, just slightly).
Need to really fix that, cause that's one of the more important parts of my project.

In AVISynth, I use ConvertToY8(). Not sure if it's different from Grayscale()

[sneaker]

What worries me slightly more is that after conversion I have "10 frames" of nothing added to the front of the output video. and the audio has become slightly out of sync (not by 10 frames, just slightly).
Need to really fix that, cause that's one of the more important parts of my project.

I cannot really comment on that because you didn't reveal all details of your workflow. Among them audio encoding and muxing process. And how you measured "10 frames". Are those "10 frames" visible in the VirtualDub preview before encoding?
Note that some audio formats (one of them being AAC) have inherent audio delay. It can be compensated for either during muxing or during audio encoding.

In AVISynth, I use ConvertToY8(). Not sure if it's different from Grayscale()

They work differently but that may or may not affect the final output.

Well, tried crf15 - still losing quality, but definitely much better.
I'll try 7, or something.

And encoding to Xvid looks better? crf 15 is relatively low already, so that surprises me.

[raffriff42]

For re-editing, your best choice among supported codecs is ProRes. A .vdprof file for it is given here.

[Cornucopia]

I agree, raffriff42. Especially since that is what FCX would need to transcode to anyway for editing. It may import h264+AAC-in-MP4 but that's not how it's going to work if you want to edit smoothly in FCX.

Going direct saves 1 stage of degradation.

Scott

[a1s2d3f4]

Thanks to everyone's replies.
On the subject of ProRes, the file sizes are huge, and I don't see visually any improvement on my much compressed XVID output. It is so much easier to upload smaller files. I want to try to troubleshoot this H.264 route first before giving up.

To give more details on my project.
I rip old b&w documentaries from SD DVDs. I then build an AVIsynth script that loads them in, does some srestore() (to get back to 24fps original films), then I do frame interpolation, etc etc.
The result is an AVS file that I can load and view in VirtualDub.

This is my reference point. It looks as good as it's going to.
I would typically export it out as an XVID AVI. As I alluded above, it looks pretty good. Visually, the pixels seem to match when I look at the Virtualdub output panel with the original .avs showing, and then another virtualdub with the XVID AVI loaded. Same frame, same image.

With using raffriff's external encoder scripts, the H.264 might be a way forward. I was able to generate a much better looking output when setting -crf to 10. It's still making it slightly "soft", but then again, the file size was only 664MB, as opposed to 843MB that I get with top quality XVID export. So, now I am waiting to see what -crf 7 would get me. Maybe I should just do -crf 1 already

Anyway, this issue of "added frames", audio sync.
This is what I see, and I don't know what it implies:

If I take the H.264 .mov output and then open it in VirtualDub, using whatever plugin I have for doing that, I get this weird effect where the first 10 frames appear to be "dumb frames", I don't know how to describe it. My video begins with a fade in, so the first frame is black. So, I have to step through 10 frames of this mov before the image starts to fade in. Also, comparing frames between my original .avs and this output, I need to be "10 frames off" for them to match, hence my "10 frames are added" observation.

It might be that this file is read incorrectly by VirtualDub. My total frame length for original AVS (as shown in Vdub timeline) is 37466. Length for h.264 mov is also 37466. So that seems to suggest there isn't a problem. But again, frame 37465 of the .MOV is matched by frame 37455 of my .AVS.

Also, there is the sound out sync issue when playing the file in VDub. It's not the case when I use other players (like VLC).
And, when VLC gets to the end of my .MOV, it fades out completely, unlike VirtualDub where it gets to "10 frames before the end" and stops.

So, any ideas? Maybe all is well, and I need to figure out how to fix VirtualDub's ability to correctly import MOVs with H.264?

[smrpix]

To echo what cornucopia said, it really doesn't matter what looks good enough to you or what the file size is for now -- what matters is what FCPX can work with. After the FCPX editing, it can be compressed to whatever you like.

Also, if your audio is off, it can easily be slipped in FCPX.

[a1s2d3f4]

@smprix I do understand the point about it being "for now". It's just that if I can get away with H264, it makes it easier on my end.
I use these video files for my own purposes too, and I like for them to be immediately playable on my machine, in sync (which seems to be ok in everything BUT VDub) and, if possible, since they live on my machine (or, rather, the external drive), I'd like them to take up less space. I do want it to work right away after being slipped into FCP - no audio adjustments needed.

[sneaker]

If I take the H.264 .mov output and then open it in VirtualDub, using whatever plugin I have for doing that?

That not very specific, now, isn't it? Why don't you write an AviSynth script with ffms2 or l-smash for opening?

[raffriff42]

If you are doing a complex project, consider that ProRes should be much more responsive in Final Cut.
You won't really notice until late in the process, with lots of edits on the timeline...

If you must use h264, you can get somewhat better performance with preset=veryfast (fewer lookahead & b-frames) and keyint=30.
Using both options will increase file size by about 30% (guessing)

[a1s2d3f4]

If I take the H.264 .mov output and then open it in VirtualDub, using whatever plugin I have for doing that?

That not very specific, now, isn't it? Why don't you write an AviSynth script with ffms2 or l-smash for opening?

I don't remember exactly how I "taught" VirtualDub to open MOV files. I think FCCHandler used to have a plugin on his/her website, which I downloaded at some point.
The thing is, I don't really know how to find out what happens when I open MOV in Virtualdub. I am assuming that the quicktime.vdplugin is somehow used, but I am not sure how to verify, so that's why I said "whatever plugin".

In regards to your suggestion about using ffms2 or I-smash in my AVISynth script, I am not sure why it should matter for exporting to H264. I happen to use DGDecode_Mpeg2source after processing the VOBs with DGIndex AviSynth plugin. This part of things doesn't appear to be causing any problems, unless you know something I don't.

My concern right now is with getting H264 MOV to
a) look as good as XVID and
b) not take up too much space (I am already at crf=7 and it is still losing some detail in frames and it's a greater file size than my trusted XVID exports) and
c) figure out how I can re-import these MOVs back into VirtualDub for frame by frame analysis and not have this weird "10 frames off" issue.

@riffraff I'll try your suggestions

[a1s2d3f4]

If you are doing a complex project, consider that ProRes should be much more responsive in Final Cut.
You won't really notice until late in the process, with lots of edits on the timeline...

If you must use h264, you can get somewhat better performance with preset=veryfast (fewer lookahead & b-frames) and keyint=30.
Using both options will increase file size by about 30% (guessing)

My friend mostly edits "around" my clip, so there wouldn't be any transitions on top of my H264 data, so I am assuming responsiveness wouldn't be much of an issue.

[sneaker]

I don't remember exactly how I "taught" VirtualDub to open MOV files. I think FCCHandler used to have a plugin on his/her website, which I downloaded at some point.
The thing is, I don't really know how to find out what happens when I open MOV in Virtualdub. I am assuming that the quicktime.vdplugin is somehow used, but I am not sure how to verify, so that's why I said "whatever plugin".

In regards to your suggestion about using ffms2 or I-smash in my AVISynth script, I am not sure why it should matter for exporting to H264. I happen to use DGDecode_Mpeg2source after processing the VOBs with DGIndex AviSynth plugin. This part of things doesn't appear to be causing any problems, unless you know something I don't.

I mean writing a new script using ffms2 or lsmash to test your .mov output. Open that script using the inbuilt file avi opener of VirtualDub. That way you bypass any such mov plugin and see if the 10 frames are just a problem of the plugin or actually of the file.

[a1s2d3f4]

I don't remember exactly how I "taught" VirtualDub to open MOV files. I think FCCHandler used to have a plugin on his/her website, which I downloaded at some point.
The thing is, I don't really know how to find out what happens when I open MOV in Virtualdub. I am assuming that the quicktime.vdplugin is somehow used, but I am not sure how to verify, so that's why I said "whatever plugin".

In regards to your suggestion about using ffms2 or I-smash in my AVISynth script, I am not sure why it should matter for exporting to H264. I happen to use DGDecode_Mpeg2source after processing the VOBs with DGIndex AviSynth plugin. This part of things doesn't appear to be causing any problems, unless you know something I don't.

I mean writing a new script using ffms2 or lsmash to test your .mov output. Open that script using the inbuilt file avi opener of VirtualDub. That way you bypass any such mov plugin and see if the 10 frames are just a problem of the plugin or actually of the file.

Got it. Yes, doing something like DirectShowSource(<my mov file>) results in correct showing in VirtualDub. I guess, there is something about H264 MOV files and FCCHandler's plugin.

So that leaves me having to deal with a) + b) issue from the previous post.
Could it be that XVID algorithm is just an overall better algorithm than this H.264 thing I am getting from ffmpeg? Might a different ffmpeg version result in a better H.264 output?

[sneaker]

Could it be that XVID algorithm is just an overall better algorithm than this H.264 thing I am getting from ffmpeg?

You'd be the first one to find Xvid better than x264 in like 10 years. But maybe your tastes are .. special. Upload samples that show the differences you speak of. Also try tuning (-tune film or -tune grain for starters).

Might a different ffmpeg version result in a better H.264 output?

No, not unless your ffmpeg is outdated.

[a1s2d3f4]

Could it be that XVID algorithm is just an overall better algorithm than this H.264 thing I am getting from ffmpeg?

You'd be the first one to find Xvid better than x264 in like 10 years. But maybe your tastes are .. special. Upload samples that show the differences you speak of. Also try tuning (-tune film or -tune grain for starters).

Might a different ffmpeg version result in a better H.264 output?

No, not unless your ffmpeg is outdated.

I have just tried to do careful comparison analysis between my source exported as H264 and XVID. Actually, both are about the same. XVID is not "sharper", like I thought.
There is certainly some degradation of quality in some areas of the frame which I didn't use to see until this cross comparison.
Right now my H264 at cfr 7 results in a 20% larger file size than my XVID for about the same quality of image.

If you are still interested, I'll upload PNGs of frame captures later. One way to compare is to flick back and forth between the two images in the same image viewer.
Another analysis could be to do pixel value subtraction and actually see the result as a separate image file.

I'll give those tune parameters a try too.

[Cornucopia]

Regardless of what YOU want to do, you are collaborating with another who is dependent on your material. You aren't being a good helper by giving h264, much less xvid. FCX is based on AVFoundation and that platform only accepts 3 codec formats natively: prores, h264, and uncompressed, with a strong preference on prores.
ALL other codec formats must be converted (non-optimally) to one of those, and all other containers must be converted to MOV.
And since there are multiple flavors of h264 you may not even have a flavor that is compliant enough to be accepted without reconversion.

Yes, the files will be huge. They're supposed to be. Get over it. This should be a no-brainer single convert to prores.

Scott

[a1s2d3f4]

Regardless of what YOU want to do, you are collaborating with another who is dependent on your material. You aren't being a good helper by giving h264, much less xvid. FCX is based on AVFoundation and that platform only accepts 3 codec formats natively: prores, h264, and uncompressed, with a strong preference on prores.
ALL other codec formats must be converted (non-optimally) to one of those, and all other containers must be converted to MOV.
And since there are multiple flavors of h264 you may not even have a flavor that is compliant enough to be accepted without reconversion.

Yes, the files will be huge. They're supposed to be. Get over it. This should be a no-brainer single convert to prores.

Scott

I sent him a short H264 snippet (one of my earlier bad-looking attempts) and he said it imported just fine, so my guess is that H.264 from ffmpeg is compatible.

ProRes is not a problem except that I don't have the Google Fiber-type bandwidth to upload these huge files quickly, and I don't yet own lots of cloudspace to store huge amounts of data (I don't even have that much space for all my clips on my local USB harddrives, not to mention the fact that my laptop's USB Controller has hit its limit with USB devices....). I might change my mind in the future about this, but, for multiple reasons, I am inclined to use something that takes up less space if in a pixel by pixel comparison of individual frames there is very little advantage in the ProRes over H264.
Again, I'm fully aware of the difficulties the CPU will have in processing H264 in FCP, vs ProRes. That's why I mentioned that in these projects my videos will be largely untouched except for maybe an additional fade in or fade out at the end.

[Cornucopia]

Ok if you say so. But the h264 encoder IS doing more change to the image (vs protest) than might be visible to the unaided naked eye. Color sub-sampling, for starters.

Just a friendly professional warning...

Scott

[a1s2d3f4]

Ok if you say so. But the h264 encoder IS doing more change to the image (vs protest) than might be visible to the unaided naked eye. Color sub-sampling, for starters.

Just a friendly professional warning...

Scott

I don't disagree.
If a project needed to be nothing short of top quality, I'd have to use ProRes (preferably, the lossless kind).
Color sub-sampling: does it really apply since my footage is b&w?

[Cornucopia]

No i guess not

[sneaker]

And "ProRes" can be sub-sampled while H.264 can also do without sub-sampling. But I don't know what FCP X supports - at least AVC‑Intra 4:4:4 is supported according to their website.