Saving space with MKV H.264 files

[Selur]

totally strange these values should not interfere with each other,..

[Ahari]

Yayy!!

The latest PM'd copy works perfectly. The autocrop, reset problem is fixed!! Great work!

Now I can play again.

PS: I have abandoned the auto interlacing analysis and everything works much faster now! As you said in a PM, I won't need it for most of my stuff as I mostly use rips from original DVDs and not recoded from TV where the interlacing flags can get mixed up. Btw it seemed to be working well before I turned it off.

[Ahari]

I just noticed that if I set "Auto deinterlacing" and select the Deinterlacer as "Yadif (Avisynth)" with mode 1 and 'order' -1 and then save the global profile, this setting is back to default mode 0 and order 1 when I reload the saved global profile.

This is now the only problem (so far) that is still not working.

I will continue to play later. Gotta go out now...

I just did a quick check before going out with throwing a folder with 2 videos in it at Hybrid and it still only autocrops the first video properly - the second video is only cropped at the top.

[Selur]

please create a debug output of the process (just enable the debug output before loading the folder)

Cu Selur

Ps.: I know what is causing the Yadif (Avisynth) problem, but it will take a while to change it

[Ahari]

Here is the debug log for the folder with 2 files.

http://dl.dropbox.com/u/3334489/Hybrid/HybridDebugOutput%20%286%29.txt

Btw: I don't know what you did, but the latest PM'd Hybrid's global profile seems to be remembering the Yadif (Avisynth) settings! I will try to break it, but it seems to be working well! You said I should use the Yadif custom (Avisynth) one in the meantime, but I didn't need to change it.

[Ahari]

Btw: I don't know what you did, but the latest PM'd Hybrid's global profile seems to be remembering the Yadif (Avisynth) settings! I will try to break it, but it seems to be working well! You said I should use the Yadif custom (Avisynth) one in the meantime, but I didn't need to change it.

No, I checked it and it is just broken. It seems to remember the settings now, but doesn't apply them to the video. Settings also disappear completely! I will try Yadif custom as suggested until a fix is made.

What is the difference with Yadif custom?

[Selur]

Yadif -fast => Yadif through MPlayer/ffmpeg (FieldOrder will be set by Hybrid; no settings for user to change stuff)
Yadif - custom => Yadif through MPlayer/ffmpeg (FieldOrder&Mode should be kept)
Yadif (Avisynth) => Yadif through Avisynth (FieldOrder will be set by Hybrid; no settings for user to change stuff)
Yadif - custom (Avisynth) => Yadif through Avisynth (FieldOrder&Mode should be kept)

also redownload the version (old link should work), I changed some small stuff yesterday

[Ahari]

Thanks - will do.

I am busy using the last version. Seems to be working nicely in general. I am using the Yadif-custom (Avisynth) filter you recommended and it seems to be working well. Thanks.

Btw: I notice that the previews are not the same as the actual output file. Is there any way to get a full preview that will be the same as the output? It is very difficult to adjust settings for things like DeBlock QED and sharpening without a live preview or at least a full preview that will match the output. At the moment I am guessing the settings and processing a small clip, checking the results and guessing again! Maybe I am missing something.

[Selur]

I notice that the previews are not the same as the actual output file. Is there any way to get a full preview that will be the same as the output?

Not the same in what regard?
Hybrid takes the source and should apply all the filters you use. It can't simulate the effect of the compression you apply. (there's no other way than to apply the compression,...)

Or do you think Hybrid is not applying all the filters you selected?

Cu Selur

[Ahari]

In this case, I am taking some old home videos from VHS tapes, cropping the noise off the edges, resizing back to original dimensions, doing a bit of deblocking and sharpening just a little to compensate for the softening from deblocking and saving as an MP4 using x264.

The resultant video looks a lot softer than the preview and the only way to evaluate the settings fully is to actually process a clip of the video.

It would be really nice to be able to see the actual effects of everything in a live window (sort of like Avidemux). Don't get me wrong, I think Hybrid is way better than Avidemux, but it does have some nice features that make it easier to "turn the knobs and tweak the settings".

Or do you think Hybrid is not applying all the filters you selected?

Hybrid appears to be doing everything I ask of it.

[Selur]

afaik Avidemux too only processes the source video and does not show any sort of precomputed compressed video preview,...

[Ahari]

It does have a nice side by side preview window that shows the results of the filters. I'm not sure if it previews the actual compression settings, but it works well and is easy to use. I only mention it because I like the way it works.

It would be nice to have something similar in Hybrid that could preview the compression settings even if only for a chosen frame. Something that would apply all the settings including the compression as you adjust them. That way you could see the effect of fiddling immediately as you play with the settings. Just an idea...

[Selur]

It does have a nice side by side preview window that shows the results of the filters.

There side by side view is basically the same as Hybrids "Filter View" (or Avisynth Preview with 'Filter view' turned on) they do not do not allow to preview anything with the compression settings. (I checked the source code.) And they only apply them after you select them, there are not doing anything like an instant preview.
Main difference is a design question, they always show the preview or side-by-side view and incorporated it into the GUI and Hybrid choose not to.

Something that would apply all the settings including the compression as you adjust them.

What you are looking for is more like an NLE tool, like Edius, Avid and similar

-> Sorry, but I don't think that I'll incorporate the preview as a fixed view into Hybrid.

Cu Selur

[Ahari]

There side by side view is basically the same as Hybrids "Filter View" (or Avisynth Preview with 'Filter view' turned on) they do not do not allow to preview anything with the compression settings. (I checked the source code.) And they only apply them after you select them, there are not doing anything like an instant preview.

Cool, so I'm not missing out on anything major by sticking with Hybrid.

Main difference is a design question, they always show the preview or side-by-side view and incorporated it into the GUI and Hybrid choose not to.

I suppose that is your choice! The preview as you have it works well anyway - I just thought it would be nice not to have to click the preview button all the time.

What you are looking for is more like an NLE tool, like Edius, Avid and similar

Wow - expensive stuff!! I have a nice little editing program (PowerDirector) that does what I need in that regard. Hybrid just does the more technical stuff much better!! I like Hybrid for getting the videos ready for later processing into a movie if I want. Mostly, my Hybrid processed stuff will stay as is without any further editing - my video archive.

-> Sorry, but I don't think that I'll incorporate the preview as a fixed view into Hybrid.

Pity! It would be nice even as an option to turn on or off. Not too serious, I really like Hybrid anyway

The main reasons for me looking to Hybrid instead of Avidemux is Hybrid is much easier to tweak the settings and batch process lots of files (when the folder problem is fixed ) and Avidemux is kinda stalled - it takes forever for anything to get fixed. You are obviously passionate about Hybrid and that makes it a pleasure to play with!!

[Ahari]

Selur, I am finally having some fun with Hybrid. Thanks for all your efforts so far - I hope anyone reading through this thread will learn as much as I have.

My head is spinning a bit less after all this, but it is still a lot to digest for a newbie! Your experience would help a lot.

Can you help me out with some suggested settings for converting my VHS videos to mp4 files using x264? The videos have to be cropped a bit and I think bobbing would be best (QTGMC or Yadif mode 1?).

I don't want to lose any quality as these videos are precious to me.

I think a bit of sharpening is in order to compensate for the bobbing and resizing - is that right? Any setting suggestions for sharpening?

Any help is much valued.

[Selur]

QTGMC or Yadif mode 1?

QTGMC (bob enabled); assuming the source is interlaced.

I think a bit of sharpening is in order to compensate for the bobbing and resizing - is that right?

No, qtgmc already does some sharpening, so unless the source needs sharpening independent of it being interlaced or not.

Depending on the quality of the input material denoising and color corrections are probably not a bad idea,.. but that you have to decide on you own. VHS captured material can give nice to really bad source material,..

[Ahari]

Thanks for the tips!

I am playing with the settings in QTGMC with some really good results!

Hybrid is rocking!

Any progress with the autocrop not working properly when a folder is processed and the problem with no audio in the previews?

[Selur]

Any progress with the autocrop not working properly when a folder is processed and the problem with no audio in the previews?

Audio&Preview: if you want audio don't use Avisynth.
Cropping: works fine here -> need debug output or better two files that cause the problem,..

[Ahari]

Cropping: works fine here -> need debug output or better two files that cause the problem,..

Here is the debug.
http://dl.dropbox.com/u/3334489/Hybrid/HybridDebugOutput%20%287%29.txt

It crashed this time.

Files are a bit big to post.

[Selur]

okay,...

1. make sure the avisynth script ("C:\Users\Alan\AppData\Local\Temp\encodingTempAvi synthSkript_12_32_09_901.avs") still exists.
If not create a .avs text file with the following content:

SetMemoryMax(768)
SetMTMode(5,8)
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\ffms2.dll")
LoadCPlugin("C:\Program Files\Hybrid\avisynthPlugins\yadif.dll")
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\mt_masktools-26.dll")
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\DctFilter.dll")
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\deblock.dll")
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\RepairSSE2.dll")
LoadPlugin("C:\Program Files\Hybrid\avisynthPlugins\RemoveGrainSSE2.dll")
Import("C:\Program Files\Hybrid\avisynthPlugins\Deblock_QED.avs")
Import("C:\Program Files\Hybrid\avisynthPlugins\Dehalo_alpha_mt.avsi")
FFVideoSource("F:\My Videos\Mede8er\Working\Hybrid\HAPPY_FEET_TWO\title04.mkv",cachefile="C:\Users\Alan\AppData\Local\Temp\title04_12_32_09_901.ffindex",fpsnum=25,threads=1)
SetMTMode(2)
ConvertToYv12(interlaced=true, matrix="PC.709")
Yadif(1,-1)
Crop(0,68,720,438)
Spline16Resize(720,576)
Deblock_QED(quant1=24,quant2=26,aOff1=1,bOff1=1,aOff2=2,bOff2=2,uv=3)
DeHalo_alpha_mt(rx=2,ry=2,darkstr=1,brightstr=1,highsens=50,lowsens=50,ss=1.5)
Removegrain(mode=2)
distributor()
return(last)

2. make sure the index file ("C:\Users\Alan\AppData\Local\Temp\title04_12_32_0 9_901.ffindex") still exists
if not recreate it with:

"C:\Program Files\Hybrid\avisynthPlugins\ffmsindex.exe" "F:\My Videos\Mede8er\Working\Hybrid\HAPPY_FEET_TWO\title04.mkv" "C:\Users\Alan\AppData\Local\Temp\title04_12_32_09_901.ffindex"

3. Check if the decoding part works:

"C:\Program Files\Hybrid\mencoder.exe" -lavdopts threads=8 -of rawvideo -o test.raw -ovc raw -noskip -vf scale,format=i420,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Alan\AppData\Local\Temp\encodingTempAvisynthSkript_12_32_09_901.avs"

(delete test.raw)

4. Assuming the decoding did work try:

"C:\Program Files\Hybrid\mencoder.exe" -lavdopts threads=8 -really-quiet -of rawvideo -o - -ovc raw -noskip -vf scale,format=i420,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Alan\AppData\Local\Temp\encodingTempAvisynthSkript_12_32_09_901.avs" | "C:\Program Files\Hybrid\x264.exe" --preset slower --crf 20 --profile high --level 4.1 --ref 16 --bframes 5 --slices 4 --psy-rd 0.4:0 --aq-strength 0.6 --vbv-maxrate 40000 --vbv-bufsize 30000 --sar 64:45 --deblock 1:1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --fps 50 --input-res 720x576 --output "C:\Users\Alan\AppData\Local\Temp\title04_new_12_32_09_901_06.264" -

(delete title04_new_12_32_09_901_06.264)

Either it's a en- or decoding problem that causes x264 to stop working:
a. if the decoding through avisynth causes a problem step 3. should throw out some warnings/errors
b. if x264 either has a problem with the raw video input or is simply running out of memory step 4. should throw some errors/warning.

Cu Selur

Ps.: the crop values ( Crop(clip, int "left", int "top", int "width", int "height") ) look fine

[Ahari]

1. make sure the avisynth script ("C:\Users\Alan\AppData\Local\Temp\encodingTemp Avi synthSkript_12_32_09_901.avs") still exists.

Yes it does exist.

2. make sure the index file ("C:\Users\Alan\AppData\Local\Temp\title04_12_3 2_0 9_901.ffindex") still exists

It also exists

3. Check if the decoding part works:
Code:
"C:\Program Files\Hybrid\mencoder.exe" -lavdopts threads=8 -of rawvideo -o test.raw -ovc raw -noskip -vf scale,format=i420,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Alan\AppData\Local\Temp\encodingTempAvis ynthSkript_12_32_09_901.avs"
(delete test.raw)

It ran. Didn't stop! Had to ctrl C it. Deleted test.raw file. Is that correct?

4. Assuming the decoding did work try:
Code:
"C:\Program Files\Hybrid\mencoder.exe" -lavdopts threads=8 -really-quiet -of rawvideo -o - -ovc raw -noskip -vf scale,format=i420,scale,format=i420 -forcedsubsonly -nosub -nosound -mc 0 "C:\Users\Alan\AppData\Local\Temp\encodingTempAvis ynthSkript_12_32_09_901.avs" | "C:\Program Files\Hybrid\x264.exe" --preset slower --crf 20 --profile high --level 4.1 --ref 16 --bframes 5 --slices 4 --psy-rd 0.4:0 --aq-strength 0.6 --vbv-maxrate 40000 --vbv-bufsize 30000 --sar 64:45 --deblock 1:1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --fps 50 --input-res 720x576 --output "C:\Users\Alan\AppData\Local\Temp\title04_new_12_3 2_09_901_06.264" -
(delete title04_new_12_32_09_901_06.264)

It also just ran on and I had to stop it with ctrl C. Deleted the resulting file.

There were no warnings or errors.

[Selur]

Okay, since the command lines all work fine my guess is that you used up to much RAM while running Hybrid and x264 ran out of memory while processing the file.
-> if you still have the job in Hybrid, try to restart it. (step no 4 included the original command line Hybrid used and reported a crash,..)

Cu Selur

[Ahari]

Hybrid and x264 ran out of memory while processing the file.

I have 16GB of RAM!?

Had to reload the job and it crashed again!

http://dl.dropbox.com/u/3334489/Hybrid/HybridDebugOutput%20%288%29.txt

[Selur]

strange,.. I'll build a new version and send it to you that will create some additional debug output that hopefully will help hunting this bug down,..

[Ahari]

Thanks.

Just a note: I was also trying to run a large file (well not that big - about 500mb) through last night after playing with the QTGMC settings and it just stopped processing twice. Once at about 14% and the second try after a reboot (just in case) at about 18%. I left it for hours to see if it was just taking a long time to process, but did not get any further. Had to close Hybrid.

Don't know if that had anything to do with it - I was loading up the filter with lots of heavy processing options to give it a "spin". Haven't had the time to check it again today. I have not applied those settings to this job btw...

[Selur]

without any detailed infos how to reproduce this -> no clue, but be aware that Avisynth is a 32bit application an therefore can not use more than 2GB of RAM, so there might be a combination of settings that causes Avisynth to want more than 2GB of RAM an will cause it to crash.
I did test all the settings and this should normally not happen, but it could.
-> I don't think that is the cause of the problem,..

[kingaddi]

Nope,gotta add each movie to the batch,no way around it.Also if you want to keep your audio at 5.1 448kps then do a ac3 pass thru if its that rate.

Do you think it is necessary to keep both a 5.1 and 2.0 audio track or will 5.1 be enough?

[Filmfan]

Selur,

Your explanation is the best I ever seen or heard explaining the 3:2 pulldown process. Still if I could trouble you with just one question to make sure that I understand everything right. According to your diagrams all odd frames are pulled down creating 2 fields, and all even numbered frames are pulled down creating 3 fields. So shouldn't the last, (24th, an even numbered), frame creating 3 (#58,# 59, #60) fields? And those last 3 frames recombined into the last 30th frame? Again, not nitpicking, just a desire to understand the process clearly. Thanks!

Regards

[Selur]

yes, you are right, the graphic isn't correct there.

[Filmfan]

Thanks again, I appreciate your quick reply. I just feel better that I understand the process completely.

Best Regards