Best way to Re-encode Canon 5D .mov file after Deshake?

[PhotoCat]

Found something interesting:

This line has problem:
FFMpegSource2("MVI_8695.mov")

This line has no problem:
FFMpegSource2("MVI_8695.mov", threads=1)

I have no idea what threads mean but it is vy repeatable! I switched between these 2 commands 3-4 times and it shows
one has problem and the other doesn't.

[PhotoCat]

Also: this has problem:
FFMpegSource2("MVI_8695.mov", atrack=-1)

This has no problem & I got audio too
FFMpegSource2("MVI_8695.mov", threads=1, atrack=-1)

[PhotoCat]

Another observation:
The one without atrack=-1 (no audio) plays smoothly in virtualdub64 while the one with audio (atrack=-1) plays vy jerky in virtualdub64.

[PhotoCat]

Thanks all for helping out!
I am very close now to re-encoding the 5Dmk2 .mov file after deshake.

Video levels look fine inside AvsPMod and Virtualdub.

avisynth codes before going into virtualdub for deshake:

LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_8695.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )


Now I am trying to compare the export methods:

I tried:

1) x264vfw compression engine installed inside VirtualDub (gives me .avi files only)
2) FFmpeg external decoder set to H.264 invoked inside VirtualDub. (gives me .mp4 files)

A few questions:

1) What are the major difference between 1) & 2) other than the output file type? .avi versus .mp4 (I have tried to keep the parameters equal. e.g. -crf 23)
2) The x264vfw export route gives me audio while the external encoder FFmpeg route doesn't. How to get audio too from FFmpeg? Note I haven't done the 601->709 conversion yet.

FFmpeg command line is: (I used -profile:v baseline -level 3.0 to gain compatibility in android & iphones too..., after reading up more on FFmpeg)

-f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -vcodec libx264 -profile:v baseline -level 3.0 -preset medium -crf 23 -acodec mp3 "%(tempvideofile)"

Tks a lot!

[poisondeathray]

1) What are the major difference of 1) & 2) other than the output file type? .avi versus .mp4 (I have tried to keep the parameters equal. e.g. -crf 23)

x264vfw doesn't export audio actually (it can only when used with something like vdub), some more advanced options are not supported , but nothing important for "normal use" eg. qpfiles, muxer and demuxer, bunch of other things that are obscure and not used often

You can configure x264 to mux in different container than AVI by using the extra commandline option box, but it doesn't mux in audio

2) The x264vfw export route gives me audio while the external encoder FFmpeg route doesn't. How to get audio too from FFmpeg? Note I haven't done the 601->709 conversion yet.

You can configure the external encoder to encode audio with ffmpeg or other encoder if you want (e.g. neroaac, qaac ), plus a muxer like mp4box. There are a bunch of tutorials and premade scripts that people distribute, just look around for them. You don't have to use MP4, you can specify a different container

There are a few obscure options that ffmpeg libx264 doesn't support either. The "full" library is only featured with x264CLI

The missing options in either are nothing to worry about for regular use


eg. riffraff has done a bunch of tutorials, and zip files you can download . Others have contributed on this board as well

http://forums.virtualdub.org/index.php?act=ST&f=3&t=21733
http://forums.virtualdub.org/index.php?act=ST&f=3&t=18840&st=0
http://forums.virtualdub.org/index.php?act=ST&f=3&t=22055

[PhotoCat]

Thanks poisondeathray once again. Maybe I wasn't vy clear. Not trying to mux in any external audio yet.
I found that the original audio from the raw .mov file has been stripped somehow when FFmpeg was used.
Any way to preserve the original audio? Otherwise my re-encoding 5D2 video workflow would be pointless if the original audio is missing. Thanks!

[poisondeathray]

Thanks poisondeathray once again. Maybe I wasn't vy clear. Not trying to mux in any external audio yet.
I found that the original audio from the raw .mov file has been stripped somehow when FFmpeg was used.
Any way to preserve the original audio? Otherwise my re-encoding 5D2 video workflow would be pointless if the original audio is missing. Thanks!

You have to use a MOV container. Because the current opensource MP4 muxers have problems muxing in uncompressed audio

So for the scripts change it to -f mov output.mov

If you want to mux in original audio, with new video, the basic template for ffmpeg is:

ffmpeg -i original.mov -i new.mp4 -map 0:1 -map 1:0 -c:v copy -c:a copy -f mov output.mov

The "map" tells ffmpeg what streams to use

-map 0:1 says video #0, stream #1 . So that is original.mov 's stream #1, which is the audio, because stream #0 is the video

You can check with ffmpeg -i original.mov and look at the stream numbering

Similarly, -map 1:0 says new.mp4's stream #0 which will be the video

[poisondeathray]

You can batch process the muxing step if you want (e.g. all the videos in a folder)

It's easier if you have them named the same , with ".MP4" added to the deshaker exported video

e.g.
MVI_6235.MOV is the original
MVI_6235.MOV.mp4 is the deshaken version

MVI_6236.MOV is the original
MVI_6236.MOV.mp4 is the deshaken version
.
.
.
etc

for %%a in ("*.mov") do ffmpeg -i "%%a" -i "%%a.mp4" -map 0:1 -map 1:0 -c:v copy -c:a copy -f mov "%%~na.new.mov"
pause

This would take all those videos in the folder, and mux the original audio with the deshaken video. The new video would take on the same name, with <name>.new.mov

[PhotoCat]

Wow it worked! poisondeathray u r such a master in video processing! The resulting .mov file has correct levels too when played
in windows media player!
It is interesting to note that the raw .mov file directly from 5D2 does not hv the correct level when played in windows media player!

Also noticed that the resulting .mov file has a slight orange tint on the skin tone. (This same orange tinting was also evident
on the .avi file exported using x264vfw.)

So next I will add the 601-709 conversion option to the FFMpeg command line switch.

Need to note something remarkable about FFMpeg:

raw .mov file size: 449Meg for 1:25mins

deshaked .avi file from x264vfw with audio (crf=23) file size: 70Meg

deshaked .mp4 file from FFMpeg (crf=23), no audio file size: 34Meg

deshaked .mp4 file from FFMpeg (crf=23), with original audio file size: 50Meg


FFMpeg rocks!

[PhotoCat]

You can batch process the muxing step if you want (e.g. all the videos in a folder)

It's easier if you have them named the same , with ".MP4" added to the deshaker exported video

e.g.
MVI_6235.MOV is the original
MVI_6235.MOV.mp4 is the deshaken version

MVI_6236.MOV is the original
MVI_6236.MOV.mp4 is the deshaken version
.
.
.
etc

for %%a in ("*.mov") do ffmpeg -i "%%a" -i "%%a.mp4" -map 0:1 -map 1:0 -c:v copy -c:a copy -f mov "%%~na.new.mov"
pause

This would take all those videos in the folder, and mux the original audio with the deshaken video. The new video would take on the same name, with <name>.new.mov

Excellent!!

[poisondeathray]

The resulting .mov file has correct levels too when played
in windows media player!
It is interesting to note that the raw .mov file directly from 5D2 does not hv the correct level when played in windows media player!

That's interesting... not sure why that would be or why there would be a difference

Also noticed that the resulting .mov file has a slight orange tint on the skin tone. (This same orange tinting was also evident
on the .avi file exported using x264vfw.)

So next I will add the 601-709 conversion option to the FFMpeg command line switch.

Yes, probably from RGB to YUV with 601, not 709 . The YUV is converted back to RGB for display with 709 for HD video

Need to note something remarkable about FFMpeg:

raw .mov file size: 449Meg for 1:25mins

deshaked .avi file from x264vfw with audio (crf=23) file size: 70Meg

deshaked .mp4 file from FFMpeg (crf=23), no audio file size: 34Meg

deshaked .mp4 file from FFMpeg (crf=23), with original audio file size: 50Meg


FFMpeg rocks!

If you want higher quality, larger filesizes, use a lower CRF value

Also if you want higher compression (smaller filesizes, higher quality) , you need to use better than "baseline" profile, because it encodes without b-frames, without cabac. Basically "baseline" turns off a lot of the important settings. Very old androids (generation 1) required baseline, newer ones can play "high" without problems. Same with iphones - only the early gen ones required baseline ,newer ones like iphone 5 can play "high" without problems

[poisondeathray]

The resulting .mov file has correct levels too when played
in windows media player!
It is interesting to note that the raw .mov file directly from 5D2 does not hv the correct level when played in windows media player!

WMP can give different results depending on what directshow splitters/decoders you have installed and configured


Can you describe how the levels are "not" correct with the original MOV ?

I'm guessing it might be reading the full range flag in the original MOV, but converting to RGB for display at standard levels . Or some variation like that . The flag isn't there in your processed files (You've taken account of the proper leves with QTInput or FFVideoSource by converting to RGB the proper Rec or PC matrix)

[PhotoCat]

The raw .mov has crushed black and blown bright while the re-encoded .mov file has perfect level.

Note my display has passed the following video test: (I had to play the mpg video)

https://forum.videohelp.com/threads/326496-file-in-Virtualdub-has-strange-colors-when-o...=1#post2022085

[PhotoCat]

The resulting .mov file has correct levels too when played
in windows media player!
It is interesting to note that the raw .mov file directly from 5D2 does not hv the correct level when played in windows media player!

That's interesting... not sure why that would be or why there would be a difference

Also noticed that the resulting .mov file has a slight orange tint on the skin tone. (This same orange tinting was also evident
on the .avi file exported using x264vfw.)

So next I will add the 601-709 conversion option to the FFMpeg command line switch.

Yes, probably from RGB to YUV with 601, not 709 . The YUV is converted back to RGB for display with 709 for HD video

Need to note something remarkable about FFMpeg:

raw .mov file size: 449Meg for 1:25mins

deshaked .avi file from x264vfw with audio (crf=23) file size: 70Meg

deshaked .mp4 file from FFMpeg (crf=23), no audio file size: 34Meg

deshaked .mp4 file from FFMpeg (crf=23), with original audio file size: 50Meg


FFMpeg rocks!

If you want higher quality, larger filesizes, use a lower CRF value

Also if you want higher compression (smaller filesizes, higher quality) , you need to use better than "baseline" profile, because it encodes without b-frames, without cabac. Basically "baseline" turns off a lot of the important settings. Very old androids (generation 1) required baseline, newer ones can play "high" without problems. Same with iphones - only the early gen ones required baseline ,newer ones like iphone 5 can play "high" without problems

Great tip for "high" with new androids! Tks!

[poisondeathray]

crushed black, blown brights suggest it's decoding at full range, but doing the RGB conversion with standard range

(analgous to how if you use ffms2, with standard "Rec" matrix to convert to RGB, instead of "PC" or full range matrix)

[PhotoCat]

crushed black, blown brights suggest it's decoding at full range, but doing the RGB conversion with standard range

(analgous to how if you use ffms2, with standard "Rec" matrix to convert to RGB, instead of "PC" or full range matrix)

I found the reason for this phenomenon: it is all in the Nvidia color settings.

There is a choice between
1) make color adjustment based on video player settings (default) and I have been using this for ages for photo editing
and
2) with Nvidia settings Full (0-255)
and
3) with Nvidia settings Limited (16-235)

I had to choose #2 to make the attached test video look right.
However, when I chose #2, the raw .mov is not looking good any more (crushed black and blown bright)

So in order to make the raw .mov look right, I have to choose setting #1. But then all the final output video will look washed out. (black
becomes grey)

Somehow I can't have the best of both worlds!

[PhotoCat]

Fortunately still pics are not affected by any of the above 3 Nvidia settings. My grey scale test pics are still intact.

[PhotoCat]

For FFmpeg external encoder, finally I have combined racer-x's x264/AVC switches with posiondeathray's color space conversion & flagging switches
as follows:

racer-x:

-f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -vcodec libx264 -level 51 -preset ultrafast -intra -crf 8 -qmin 1 "%(tempvideofile)"

posiondeathray:

-pix_fmt yuv420p -vf colormatrix=bt601:bt709 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709

The combined FFmpeg switches: (virtualdub external encoder setup)

-f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -vcodec libx264 -level 51 -preset ultrafast -intra -crf 8 -qmin 1 -pix_fmt yuv420p -vf colormatrix=bt601:bt709 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 "%(tempvideofile)"

Let's use racer-x's switches as a benchmark. The resulting video has correct levels but the color has a very slight orange tint as mentioned above.
After adding the 601 to 709 conversion switch, there seem no change in color and quality. (same slight acceptable orange tint when compared to virtualdub preview)

I am unable to compare the quality of the 2 mp4 outputs directly as I can only get one instant of WMP running.

[PhotoCat]

I have found that -crf 12 -preset medium would give me about the same file size as the original clip.

Just for sanity check, during exporting, the virtualdub log was:

input format XRGB8888
output format YUV420

[poisondeathray]

How are you viewing the output ? How are you making that determination ? Through wmp ?

What is the goal here? Those encoding settings aren't ideal for general purpose, more suited for "intermediate" format used for editing

-intra means I-frame only, so you basically lose 50% compression efficiency right off the bat

-preset ultrafast is the lowest quality preset (trade off vs. speed). It disables everything useful basically. Even xvid is better than x264 when this is used. You basically need almost 2x the filesize for similar level of "quality"

[PhotoCat]

Thanks all and especially poisondeathray for helping out.

In summary, the following workflow worked well for me in re-encoding Canon 5Dmk2 .mov file after deshake:

1) Decode Canon 5D2 .mov using avisynth & preview with AvsPMod

LoadPlugin("C:\Virtualdub32\FFMS2\ffms2.dll")
FFmpegSource2("MVI_8695.mov", threads=1, atrack=-1)
ConvertToRGB32(matrix="PC.601" )

2) Use Avsf to quick mount the .avs file in (1) so that it can frameserve to Virtualdub 64 for deshaking (I found Virtualdub 64 more robust in deshaking)

3) Deshake in virtualdub and tweak parameters for best stabilization in pass 1 and pass 2; it is important to set rolling shutter to 70% for 5D2.

4) Use FFmpeg external decoder to export the resulting .mp4 file with the following switches

-f rawvideo -s %(width)x%(height) -r %(fpsnum)/%(fpsden) -i - -vcodec libx264 -level 51 -preset medium -intra -crf 12 -qmin 1 -pix_fmt yuv420p -vf colormatrix=bt601:bt709 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 "%(tempvideofile)"

5) Copy the original audio track from original.mov to the exported .mp4 video only file:

ffmpeg -i original.mov -i new.mp4 -map 0:1 -map 1:0 -c:v copy -c:a copy -f mov output.mov

6) Yay, we are done!!

Oops... I forgot to check audio sync of the final output.mov

[jagabo]

I have found that -crf 12 -preset medium would give me about the same file size as the original clip.

That won't be the case for all videos.

[poisondeathray]

2) Use Avsf to quick mount the .avs file in (1) so that it can frameserve to Virtualdub 64 for deshaking (I found Virtualdub 64 more robust in deshaking)

Interesting that you find vdub64 "more robust" for deshaking - what do you mean by that?

Almost everyone avoids it , because it's less stable, fewer plugins available for that version



If you check the output MOV, open it with ffms2 and preview in avspmod or vdub

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec709")

or

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec601")


Do you see a difference ? And if you omit -vf colormatrix ?

[PhotoCat]

How are you viewing the output ? How are you making that determination ? Through wmp ?

What is the goal here? Those encoding settings aren't ideal for general purpose, more suited for "intermediate" format used for editing

-intra means I-frame only, so you basically lose 50% compression efficiency right off the bat

-preset ultrafast is the lowest quality preset (trade off vs. speed). It disables everything useful basically. Even xvid is better than x264 when this is used. You basically need almost 2x the filesize for similar level of "quality"

Thanks agn posiondeathray. Great info I can use

Goal is for generating a deshaked master copy of the original .mov with the highest quality possible for archive and future editing,
while maintaining about the same file size of the original .mov

It is not meant for general distribution.

So I guess I should use "slow" instead of "medium".

Would -intra gives me higher quality? If so I can take the compression hit as I am trying to create a master copy.

[PhotoCat]

I have found that -crf 12 -preset medium would give me about the same file size as the original clip.

That won't be the case for all videos.

Thanks jagabo. Just a good starting point I guess...

[PhotoCat]

2) Use Avsf to quick mount the .avs file in (1) so that it can frameserve to Virtualdub 64 for deshaking (I found Virtualdub 64 more robust in deshaking)

Interesting that you find vdub64 "more robust" for deshaking - what do you mean by that?

Almost everyone avoids it , because it's less stable, fewer plugins available for that version



If you check the output MOV, open it with ffms2 and preview in avspmod or vdub

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec709")

or

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec601")


Do you see a difference ? And if you omit -vf colormatrix ?

When I deshake with virtualdub32 in pass1, it often gets stuck in frame 60 or so and doesn't move on. So I tried Virtualdub64 with avisynth via AvsF.
It was smooth and crashed less. I will try your suggestions

[poisondeathray]

So I guess I should use "slow" instead of "medium".

Would -intra gives me higher quality? If so I can take the compression hit as I am trying to create a master copy.

No, those settings including -intra are fine for "master copy", but not "ultrafast" because it disables inloop deblocking and cabac.

You might as well use something else if you choose "ultrafast" because the quality and filesize will be better

Intra is meant for faster seeking (meant for editing in other programs), but "master copy" I-frame format usually implies a file several times larger than your original . If you wanted something similar in size to the original, with similar quality, you won't get it with I-frame only

[PhotoCat]

2) Use Avsf to quick mount the .avs file in (1) so that it can frameserve to Virtualdub 64 for deshaking (I found Virtualdub 64 more robust in deshaking)

Interesting that you find vdub64 "more robust" for deshaking - what do you mean by that?

Almost everyone avoids it , because it's less stable, fewer plugins available for that version



If you check the output MOV, open it with ffms2 and preview in avspmod or vdub

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec709")

or

FFVideoSource("output.mov")
ConvertToRGB(matrix="rec601")


Do you see a difference ? And if you omit -vf colormatrix ?

When I deshake with virtualdub32 in pass1, it often gets stuck in frame 60 or so and doesn't move on. So I tried Virtualdub64 with avisynth via AvsF.
It was smooth and crashed less. I will try your suggestions

Using the new.mov, rec709 is definitely more yellow and more red than rec601 when viewed inside AvspMod.

After omitting -vf colormatrix switch in the external FFmpeg encoder and using the newly generated .mp4 file,
similar results were observed with rec709 & rec601.
i.e. rec709 is definitely more yellow and more red than rec601 when viewed inside AvspMod.

I now can compare different files in AvspMod, as it allows me to open up multiple instances.

Comparing the 2 output files one with -vf and one without.
both used ConvertToRGB32(matrix="rec601" )

The one without -vf is slightly more orange than the one with -vf.

[PhotoCat]

Must note that all the skin tones are acceptable and it is not easy to tell the orange shift unless compared by flipping between 2 windows.

[PhotoCat]

So I guess I should use "slow" instead of "medium".

Would -intra gives me higher quality? If so I can take the compression hit as I am trying to create a master copy.

No, those settings including -intra are fine for "master copy", but not "ultrafast" because it disables inloop deblocking and cabac.

You might as well use something else if you choose "ultrafast" because the quality and filesize will be better

Intra is meant for faster seeking (meant for editing in other programs), but "master copy" I-frame format usually implies a file several times larger than your original . If you wanted something similar in size to the original, with similar quality, you won't get it with I-frame only

Tks agn poisondeathray! I will use "slow" then to create the master copy. No questions about it.

So should I omit "Intra" if I don't plan to edit directly off the re-encoded file? I would like to trade faster seek for quality (and maintain file size)
as long as in the future I can always add "Intra" to create something easier to seek or edit, without losing quality.