QTGMC Large File Problem

[Jay123210599]

I managed to deinterlace a video using QTGMC via AVIsynth and VirtualDub2, but the output video had a size of a whomping 124 GB! Is there a way to deinterlace videos and get significantly smaller output file sizes? The input video was at 5.10 GB.
[Attachment 75120 - Click to enlarge]

[Alwyn]

You probably exported to YUV uncompressed.

On the Save Video... screen, bottom left, set the video compression to one of the other codecs depending on what you want (eg lossless (HUFF, LAGS, UT video or lossy X264) and also change the Save As Type. If you use x264, you'll need to set the Audio Menu to Full Processing Mode and choose AAC or similar as the audio codec.

[magnu]

Did you set the Filesize/Bitrate Settings size properly? On Hybrid (which I use) If you are doing, for example, x264, under Encoding Mode it says Target Mode. I usually set mine to Custom and then choose either the filesize or bitrate size I want the resulting file to be.
I also use MediaInfo to find out what my original bitrate is.

I presume there are similar options in the software you have?

[Jay123210599]

These are the options for video and audio compression on VirtualDub2. I selected Direct Steam Copy for both and that's how I got the large file.
[Attachment 75124 - Click to enlarge]
[Attachment 75125 - Click to enlarge]

[Cornucopia]

Direct stream copy is for copying without needing to recompress, which works fine for simple editing (particularly I frame Gop edits). However, whenever you PROCESS video, it must ALWAYS do a full re-encode (to whatever settings you have set, or to some default like uncompressed, if you forgot to set them). It will override direct stream copy option. And QTGMC does Heavy processing (as does many AVISynth scripts).


Scott

[Alwyn]

For the video (and audio) you don't have Direct Stream Copy selected in the Video menu, because you have changed the Video compression on the Save Video screen. In Direct Stream Copy mode, you cannot change the compression: the button is greyed-out.

FFV1 does produce very large files (121MB/sec). Other lossless codecs such as HUFF or Lagarith are only around 60MB/sec. What is the purpose of your export? To further edit in an NLE, or give to others to watch? That will determine what video codec you should save with.

To explain the 5GB>124GB, the 5GB file is probably in the H264 codec, which is a highly compressed format. If you "save" that in FFV1, it's quite understandable you'll get that "huge" increase in size.

I selected Direct Steam Copy for both and that's how I got the large file.

If you had Direct Stream Copy selected, your exported file would be exactly the same as the original: 5.1GB because nothing would have been done to it.

[Jay123210599]

I used Direct Steam Copy when deinterlacing my last video and it was 124 GB in size. If I used lossless codecs, it will also be large in size?

[Jay123210599]

For the video (and audio) you don't have Direct Stream Copy selected in the Video menu, because you have changed the Video compression on the Save Video screen. In Direct Stream Copy mode, you cannot change the compression: the button is greyed-out.

FFV1 does produce very large files (121MB/sec). Other lossless codecs such as HUFF or Lagarith are only around 60MB/sec. What is the purpose of your export? To further edit in an NLE, or give to others to watch? That will determine what video codec you should save with.

To explain the 5GB>124GB, the 5GB file is probably in the H264 codec, which is a highly compressed format. If you "save" that in FFV1, it's quite understandable you'll get that "huge" increase in size.

I selected Direct Steam Copy for both and that's how I got the large file.

If you had Direct Stream Copy selected, your exported file would be exactly the same as the original: 5.1GB because nothing would have been done to it.

Here is the script for deinterlacing my last video:
DirectShowSource("title00.mkv")
QTGMC( Preset="Medium")
SelectEven()

I decided to deinterlace the video direct stream copy again for only a few minutes and it was already larger than the input.
[Attachment 75126 - Click to enlarge]
[Attachment 75127 - Click to enlarge]

[davexnet]

Here is the script for deinterlacing my last video:
DirectShowSource("title00.mkv")
QTGMC( Preset="Medium")
SelectEven()

You have to set an output codec (in vdub2, video/compression/x264, for example) otherwise you get these
massive file sizes when using an Avisynth script as your source - this is because
Avisynth always presents an uncompressed stream to the host program (vdub2)

[Alwyn]

I used Direct Steam Copy when deinterlacing my last video and it was 124 GB in size. If I used lossless codecs, it will also be large in size?

I say again, it is not possible to do a Direct Stream Copy and resave that AVS. A direct stream copy simply copies the video as-is, so at most, it will end up at ~5GB, with no processing involved.

You can check the Video menu: you will have Direct Processing mode selected.

Your "mistake" is that you are using the FFV1 codec to export your QTGMC AVS. That is giving you the large 124GB file size. If you use HUFF or Lagarith, the file size will be at least half that size.

It would be good to get the details of your original file (not the AVS). You can get them from the View menu>Text option in MediaInfo (copy and paste here), or in VDub, open the original file then go File>File information. Do a screenshot of that box and post it; that'll help with explaining what's occurring here.

Again,

What is the purpose of your export? To further edit the deinterlaced video in an NLE, or give to others to watch? That will determine what video codec you should save with.

[Jay123210599]

I used Direct Steam Copy when deinterlacing my last video and it was 124 GB in size. If I used lossless codecs, it will also be large in size?

I say again, it is not possible to do a Direct Stream Copy and resave that AVS. A direct stream copy simply copies the video as-is, so at most, it will end up at ~5GB, with no processing involved.

You can check the Video menu: you will have Direct Processing mode selected.

Your "mistake" is that you are using the FFV1 codec to export your QTGMC AVS. That is giving you the large 124GB file size. If you use HUFF or Lagarith, the file size will be at least half that size.

It would be good to get the details of your original file (not the AVS). You can get them from the View menu>Text option in MediaInfo (copy and paste here), or in VDub, open the original file then go File>File information. Do a screenshot of that box and post it; that'll help with explaining what's occurring here.

Again,

What is the purpose of your export? To further edit the deinterlaced video in an NLE, or give to others to watch? That will determine what video codec you should save with.

For others to watch, of course. Also here are the details:

General
Unique ID : 258353246348045939569265620988127380324 (0xC25D067C7E97E681F2DFF58C715C8364)
Complete name : title00.mkv
Format : Matroska
Format version : Version 2
File size : 5.11 GiB
Duration : 23 min 53 s
Overall bit rate mode : Variable
Overall bit rate : 30.6 Mb/s
Frame rate : 29.970 FPS
Encoded date : 2017-12-12 05:26:37 UTC
Writing application : MakeMKV v1.10.8 win(x64-release)
Writing library : libmakemkv v1.10.8 (1.3.5/1.4.7) win(x64-release)

Video
ID : 1
ID in the original source medium : 4113 (0x1011)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Format settings, picture structure : Frame
Codec ID : V_MPEG4/ISO/AVC
Duration : 23 min 52 s
Bit rate mode : Variable
Bit rate : 28.3 Mb/s
Maximum bit rate : 40.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 (30000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan type, store method : Interleaved fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.456
Time code of first frame : 01:00:00:0000000000
Stream size : 4.72 GiB (92%)
Language : English
Default : No
Forced : No
Original source medium : Blu-ray

Audio
ID : 2
ID in the original source medium : 4352 (0x1100)
Format : PCM
Format settings : Little / Signed
Codec ID : A_PCM/INT/LIT
Duration : 23 min 53 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 30.000 FPS (1600 SPF)
Bit depth : 24 bits
Stream size : 394 MiB (8%)
Title : Stereo
Language : Japanese
Default : Yes
Forced : No
Original source medium : Blu-ray

Menu
00:00:00.000 : en:Chapter 01
00:03:11.891 : en:Chapter 02
00:06:54.046 : en:Chapter 03
00:09:25.665 : en:Chapter 04
00:12:03.956 : en:Chapter 05
00:15:20.519 : en:Chapter 06
00:17:06.158 : en:Chapter 07
00:18:30.542 : en:Chapter 08
00:19:04.009 : en:Chapter 09
00:21:43.201 : en:Chapter 10
00:23:51.997 : en:Chapter 11

I have these selected in the video menu.
[Attachment 75128 - Click to enlarge]
[Attachment 75129 - Click to enlarge]

[Alwyn]

For others to watch, of course.

Excellent, we now know who we're cooking for. Thanks also for the file data, that helps.

I'm a novice at AVISynth, so I'm not going to promise anything, but in your script, try putting in, on the very last line,

Prefetch(x)

where x is the number of Logical processors in your CPU (Task manager>Performance tab>CPU. That might speed things up a bit.

You need to save it as an MP4 with AAC audio. Then you'll be easily able to distribute it and it will play on anything. Follow my guide here.

If the file is too big, increase the CRF (described in the guide).

Let us know how you get on.

[Sharc]

Here is the script for deinterlacing my last video:
QTGMC( Preset="Medium")
SelectEven()

Are you sure you want to baserate deinterlace it? What is your motivation of doing it?
- If the source is truly interlaced you are going to loose 50% of the temporal resolution by using SelectEven()
- If the source is telecined you should IVTC (InverseTelecine) it rather than deinterlace
If you are in doubt upload a short (3 ...4 seconds) snippet of the original file. Select a scene with motion.

[Jay123210599]

Here is the script for deinterlacing my last video:
DirectShowSource("title00.mkv")
QTGMC( Preset="Medium")
SelectEven()

You have to set an output codec (in vdub2, video/compression/x264, for example) otherwise you get these
massive file sizes when using an Avisynth script as your source - this is because
Avisynth always presents an uncompressed stream to the host program (vdub2)

What should I put in for video, compression, and x264? Will this work if I select Direct Stream Copy and won't have a large file?

[Cornucopia]

Again, to repeat what has already been explained to you:

1. PROCESSING requires access to the uncompressed form, and since it changes that uncompressed form, it is not possible to return it to a compressed form without RECOMPRESSING it. Examples of processing: Deinterlacing (including QTGMC), Resizing/Cropping, Color changes, Text subs & Picture overlays, Noise Reduction.
2. EDITING can work either on the compressed form or on the uncompressed form, depending on the original file's complexity, so sometimes requires PROCESSING, sometimes not. Examples of non-processed editing: I-frame GOP editing, Intraframe-only editing, non-destructive framerate changes incl. true ITVC.

If you choose "Direct Stream Copy" you can only do items in #2. If you choose DSC, then add items in #1, it will either change your selection to Full processing mode or it will simply ignore your setting, override it to Full processing mode, and either way will require a recompression.
If you recompress, you MUST set the compression settings. If you forget to set those settings, it will use the default, which usually happens to be UNCOMPRESSED. Which means HUGE files.

If you set your PROCESSED ("QTGMC'd") video to x264 or x264vfw in the Video settings, the usual accompanying settings would be 4:2:0 colorspace, 8bit, and then you should set the bitrate to whatever bitrate happened to match what your x264-compressed source was. (Example shown was VBR, averaging 28.3Mbps. 23m52s of that works out to: 28.3 Mbps * 1432sec = 40525.6 Mb or 5065.7 MB or 4.9469 GB).

<edit>Also, If you do QTGMC(), you don't need to (and should NOT) also do SelectEven(), as it is a waste of QTGMC and ending up with a reduced framerate with reduced quality. As @Sharc already mentioned to you.


Scott

[Jay123210599]

Again, to repeat what has already been explained to you:

1. PROCESSING requires access to the uncompressed form, and since it changes that uncompressed form, it is not possible to return it to a compressed form without RECOMPRESSING it. Examples of processing: Deinterlacing (including QTGMC), Resizing/Cropping, Color changes, Text subs & Picture overlays, Noise Reduction.
2. EDITING can work either on the compressed form or on the uncompressed form, depending on the original file's complexity, so sometimes requires PROCESSING, sometimes not. Examples of non-processed editing: I-frame GOP editing, Intraframe-only editing, non-destructive framerate changes incl. true ITVC.

If you choose "Direct Stream Copy" you can only do items in #2. If you choose DSC, then add items in #1, it will either change your selection to Full processing mode or it will simply ignore your setting, override it to Full processing mode, and either way will require a recompression.
If you recompress, you MUST set the compression settings. If you forget to set those settings, it will use the default, which usually happens to be UNCOMPRESSED. Which means HUGE files.

If you set your PROCESSED ("QTGMC'd") video to x264 or x264vfw in the Video settings, the usual accompanying settings would be 4:2:0 colorspace, 8bit, and then you should set the bitrate to whatever bitrate happened to match what your x264-compressed source was. (Example shown was VBR, averaging 28.3Mbps. 23m52s of that works out to: 28.3 Mbps * 1432sec = 40525.6 Mb or 5065.7 MB or 4.9469 GB).

<edit>Also, If you do QTGMC(), you don't need to (and should NOT) also do SelectEven(), as it is a waste of QTGMC and ending up with a reduced framerate with reduced quality. As @Sharc already mentioned to you.


Scott

In other words, I should select a video compression format in VirtualDub2 because selecting Direct Stream Copy will only use the uncompressed video as the output.

[Alwyn]

What should I put in for video, compression, and x264?

It's all in my guide I linked to in post #12.

[Jay123210599]

What should I put in for video, compression, and x264?

It's all in my guide I linked to in post #12.

I meant in the script for QTGMC.

[Alwyn]

Arr, OK.

[davexnet]

Select x264 8bit/configure/load defaults.
Start there

[Alwyn]

Jay, just to confirm, the process we're talking about is:

Open your AVS (containing the QTGMC function) in Virtual Dub. On the Video and Audio menus, choose "Full processing mode", otherwise nothing will be changed in your video.

Do your editing if needed, then do a File>Save Video...

Then you set up the x264 parameters and convert your video to h264/MP4.

You don't convert to H264 in the script itself.

[rgr]

Here is the script for deinterlacing my last video:
DirectShowSource("title00.mkv")
QTGMC( Preset="Medium")
SelectEven()

1. You will get a better effect by using preset = slow or slowest with QTGMC
2. Why SelectEven? This removes half the frames.
3. Use the x264 codec for encoding and Full Processing Mode, not DirectStreamCopy.