How could I do a very fast conversion from MPG skipping video frames?

[falco2000]

Hello, this is my first post on this forum

I need to convert on my old and slow laptop many 1GB MPG files to the lowest possible filesize to archive and replay them in the future just to catch the plot of the movies (they are all legally recorded from free digital TV for personal use).

I'm using Handbrake to convert these MPG files to MP4+H.264 files, reducing frames from 720x576 pixels to 160x120, video bitrate from thousands of Kbps to 100Kbps, and audio bitrate to 70 Kbps. The result is acceptable for me, but the time to get the output file is really too long (almost the duration of the movie itself).

I've seen using Xvid4PSP and watching its bar indicators ("buffer occupancy": decoding, filtering, encoding, writing) that this problem seems to be with the decoding part of the conversion. It seems to me that's the decoding of the original movie that takes a long time, not the encoding in MP4+H.264. But I could be wrong. Anyway...

... here's the question: is there a way to drastically reduce the conversion time, for example forcing the "I don't know which" conversion free program to take and convert only 1 video frame every 10 (I don't need the output movie to be 30 or 25 fps smooth), leaving the audio "as is" instead (otherwise the dialogues would be unintelligible)?
Or any other method or decoder that speeds up the conversion?

I hope I clearly explained my problem.

Any answer is appreciated, thank you very much!
Falco



PS: have a look at my profile to see my computer details.

[falco2000]

In the meanwhile, I tried to set Handbrake output movie framerate to the minimum, that's 5fps (there's a drop-down menu that has only fixed values).

The original movie was 1.8 GB, about 1h:40mins of movie at 25fps and 720x576 pixel large, video at 15000 Kbps and audio stereo at 128 Kbps 48KHz.

With Handbrake I changed or lowered some characteristics: two output MP4+H.264 files, 160x120 pixel large, video at 100Kbps (but Windows says 99 & 173) and audio at 74Kbps 44KHz, but with different framerates:

• 25 fps - total file size: 130 MB - total time for conversion: 1 hour and 7 minutes
• 5 fps - total file size: 128 MB - total time for conversion: 30 minutes

I reduced by half the conversion time (and that's a good news, although my goal is to convert in much less minutes, let's say 5 or 10).

But the file size has remained the same. How is it possible?
Doesn't the passage from 25 fps to 5 fps mean that I have 5 pictures every second instead of 25? This should have reduced the total size by 5 times, that means that the file size should be about 130 MB divided by 5, that equals to 26 MB. Why didn't that happen?

It's just the thing I want to obtain!

[jagabo]

The audio compression is probably your bottleneck. Most audio encoders are not multithreaded. Or not well multithreaded. Try keeping the original audio (or no audio at all) and see how much faster the process is.

Using an Intel i5 2500K I tried a similar conversion using DgMpgDec, AviSynth, and the x264 CLI encoder at the slow preset (single pass, CRF=18), I was able to encode 44 minute VOB file to a 18 MB MKV file in about 90 seconds (switching to the veryfast preset didn't make much difference) with less than 50 percent CPU usage. That didn't include the initial building of the index with DgIndex (about 30 seconds) or audio processing though. Using DirectShowSource() (and hence ffdshow as the decoder) instead of Mpeg2Source() (DgMpgDec) reduced the time to about 35 seconds. Muxing in the original AC3 audio with MMG took about 15 seconds to set up and 1 second to perform.

General information:

Use faster x264 settings for faster encoding. At fixed bitrates you will get lower quality though. But with your tiny frame size and low frame rate encoding probably isn't your bottleneck.

When using bitrate based encoding:

 file size = bitrate * running time

regardless of the properties of the video. If you want a smaller file use a lower bitrate.

If your source is interlaced or telecined, deinterlacing or inverse telecine may be slowing you down. If you don't care about video quality disabling those will speed up the process.

Other processing of your source may be slowing you down too. Deblocking and deringing increase the quality but won't make much difference since you're downsizing to 160x120. Disable all filtering of the source.

[jman98]

This thread takes the cake as the most ridiculous request for help I've ever seen here. To me this is the equivalent of someone deciding it would be a great idea to smash his own nuts in with a hammer. If you actually think videos like that are worth making AND watching, I would just have to say that your personal preferences are bordering on the crazy.

It is a very common mistake in the video field that inexperienced people make in that they assume that fewer frames will lead to smaller file sizes. Back 10+ years ago it was a lot more common to see really crazy crap like 12 fps video because someone thought it would lead to smaller files. Also, there is a similar misconception in audio that the lower the sampling rate, the smaller the file will be, which is also incorrect.

[falco2000]

This thread takes the cake as the most ridiculous request for help I've ever seen here. To me this is the equivalent of someone deciding it would be a great idea to smash his own nuts in with a hammer. If you actually think videos like that are worth making AND watching, I would just have to say that your personal preferences are bordering on the crazy.

It is a very common mistake in the video field that inexperienced people make in that they assume that fewer frames will lead to smaller file sizes. Back 10+ years ago it was a lot more common to see really crazy crap like 12 fps video because someone thought it would lead to smaller files. Also, there is a similar misconception in audio that the lower the sampling rate, the lower the file will be, which is also incorrect.

Thank you jman98, I'm not as intelligent and cultured as you, congratulations. Next time please avoid answering me if you have to judge me, my knowledge and what I want to obtain (that's what I need and requested help for in this forum). Have a good day. Goodbye.

PS: where I live people usually say "No one was born already taught".

[AlanHK]

I need to convert on my old and slow laptop many 1GB MPG files to the lowest possible filesize to archive and replay them in the future just to catch the plot of the movies.

Copy the plot summary from IMDB, save a lot more space and time.
IMDB usually has a trailer too.
Faster and a lot easier on the than a jerky low res conversion.

[DB83]

Well. Confession time. I was in that class that thought that small frame-size plus low fps would give you small files but....

I stll have some of those postage stamp vids and sure wish I had the power of foresight as once you take something out you can not get it back. Or to put it another way, sometime down the line you may wish you had larger size vids and without the original source you are shot.

[falco2000]

I need to convert on my old and slow laptop many 1GB MPG files to the lowest possible filesize to archive and replay them in the future just to catch the plot of the movies.

Copy the plot summary from IMDB, save a lot more space and time.
IMDB usually has a trailer too.
Faster and a lot easier on the than a jerky low res conversion.

The plot summaries on IMDB, TV.com, Wikipedia and others are usually very short (1, 2 or 3 sentences) just to give the idea of what happens in a TV series episode, it's not the entire plot. And even when they are longer, the "finale" is usually not included, so if I can't remember it, the plot summary doesn't help.

Anyway, please don't judge my will of having a nano-version of my video recordings. If you know how to help me, please give me your suggestions, I'll be grateful for them, and I'll learn something new.

If you want to tell me that I'm crazy, instead, please skip this thread and do not reply. Nobody forces you to write.

Thanks a lot again.

PS: another way to help me is to donate me some thousands dollars, so I'll be able to buy many 16 TeraBytes hard drives and I won't need any conversion.

[jagabo]

I'm using Handbrake to convert these MPG files to MP4+H.264 files, reducing frames from 720x576 pixels to 160x120, video bitrate from thousands of Kbps to 100Kbps, and audio bitrate to 70 Kbps. The result is acceptable for me, but the time to get the output file is really too long (almost the duration of the movie itself).

I just tried this with Handbrake and a 20 minute VOB file, converting to h.264/aac/mp4 (keeping the original 25 fps). It took about 1 minute. I used CRF encoding rather than 2-pass.

[Trippedout]

I used winavi all in one converter (pros: very fast converter F1 fast) (con: not free but free to try)
original file size was 6.47 gig converted file size 897mb video preveiw exceptable setings meduim quality
mpg frame width 384 hight 304 conversion time 6.49secs not sure if this is what you mean but there you go
all the best

[gregalan]

This thread takes the cake as the most ridiculous request for help I've ever seen here. To me this is the equivalent of someone deciding it would be a great idea to smash his own nuts in with a hammer. If you actually think videos like that are worth making AND watching, I would just have to say that your personal preferences are bordering on the crazy.

It is a very common mistake in the video field that inexperienced people make in that they assume that fewer frames will lead to smaller file sizes. Back 10+ years ago it was a lot more common to see really crazy crap like 12 fps video because someone thought it would lead to smaller files. Also, there is a similar misconception in audio that the lower the sampling rate, the smaller the file will be, which is also incorrect.

If you use crf fewer frames will mean smaller file size, it's when using a constant bitrate that the file size remains the same.

You're also wrong about audio, lower sampling rate does lower the size of a wav file.
If you meant compressed audio the same applies about constant bitrate vs quality based vbr .

You need to think before you dismiss people as "crazy".

[Cornucopia]

This thread takes the cake as the most ridiculous request for help I've ever seen here. To me this is the equivalent of someone deciding it would be a great idea to smash his own nuts in with a hammer. If you actually think videos like that are worth making AND watching, I would just have to say that your personal preferences are bordering on the crazy.

It is a very common mistake in the video field that inexperienced people make in that they assume that fewer frames will lead to smaller file sizes. Back 10+ years ago it was a lot more common to see really crazy crap like 12 fps video because someone thought it would lead to smaller files. Also, there is a similar misconception in audio that the lower the sampling rate, the smaller the file will be, which is also incorrect.

If you use crf fewer frames will mean smaller file size, it's when using a constant bitrate that the file size remains the same.

You're also wrong about audio, lower sampling rate does lower the size of a wav file.
If you meant compressed audio the same applies about constant bitrate vs quality based vbr .

You need to think before you dismiss people as "crazy".

No, you may be under the false assumption that the audio or video is uncompressed. With (lossily) compressed media, the bitrate/compression encoding parameters, whether for CBR or VBR or CRF/CQ, are the ONLY determiner of the overall average bitrate. And, as has been quoted so often, the rule is "BITRATE * LENGTH = FILESIZE". That is a cardinal rule.

You are also not taking into consideration the fact that CRF/CQ still ultimately applies a variable bitrate which has an "AVERAGE". And that AVERAGE is the ONLY determiner of filesize. And the only thing that determines the variable bitrate is the CRF value and the "quality/complexity" of the media. Change the CRF value, the compression will change, the variable bitrate will change, the average bitrate will change and so finally the filesize will change. But "quality" can be influenced (not in a one-to-one relationship though) by framerate/samplerate.

So, if you count something that has a 3rd or 4th generational influence on filesize, then yes it makes a difference, but it's not a difference you can/should count on, because it's at the mercy of the algorithm that decides what the quality/complexity is. And like I said, it's not a 1:1 linear change. Others will back me up here - with a drop in framerate/samplerate of 1/2 of the original, one would expect a drop in 1/2 of the bitrate, but that is not so. The reason why is the fact that while there are 1/2 less frames to deal with, and so it is less complex, there is greater difference BETWEEN frame-to-frame (or sample-to-sample), so more bits have to be applied to account for the greater difference. It works the same in the reverse with higher framerate/samplerate/bitdepth media files, you get better "efficiency" because of less data entropy.

Getting back to the uncompressed stuff, most noobs treat audio & video as if they were uncompressed even when they are in actuality lossily compressed. This is where most of those fallacies arise.

Also, your argument of lossily compressed files only holds water at all when dealing with CRF/CQ files. This is still the minority of files. The great majority of audio encoding is still CBR, even VBR is small in comparison. And ALL pro video compression done specifically for use with DVD or BD or for broadcast or streaming use either VBR or CBR, not CRF.

Scott

[falco2000]

It is a very common mistake in the video field that inexperienced people make in that they assume that fewer frames will lead to smaller file sizes. Back 10+ years ago it was a lot more common to see really crazy crap like 12 fps video because someone thought it would lead to smaller files. Also, there is a similar misconception in audio that the lower the sampling rate, the smaller the file will be, which is also incorrect.

I caught the concept. If the encoding operation is based on a "bit per second" function, that is what determines the file size.

But I did a lot of tests, and:

1) keeping the original high bitrate and lowering the frame rate creates a video with a much better image quality (since there are less frames in a second to use the, for example, 25 Kbps "available" data). If I keep a 25 fps, instead of reducing it to 5 fps (I didn't succeed in obtaining lower framerates, for ex. 1fps), the video is obviously more fluent, but the image is horribly full of blocks.
More over, the encoding is much faster, because there are less frames per seconds (i.e. data) to elaborate (5 vs. 25)

2) the same concept happens keeping the original high bitrate and lowering the frame size, which creates a video with a much better image quality (because there are less pixels to encode in each frame using, for example, 25 Kbps "available" data). If I keep 720x576 pixels, instead of reducing it to 160x120, each frame/picture is much larger, yes, but it's again horribly full of blocks and artifacts. At 160x120 pixels resolution, instead, playing it in VLC for example, and enlarging the movie with the 2x zoom option, or enlarging the VLC window to approximately match the 720x576 pixels size (with movie fit to window, obviously), the picture quality is incredibly better: smooth pictures, yes, no fine details, but also not a mess of blocks like in the 720x576 px version of the movie.
More over, as for point 1), the encoding is much faster, because there are much less pixels (i.e. data) to elaborate.

3) The same concept applies to audio: I left the original 48KHz (lowering to 44KHz seemed not so relevant to me, and I never considered lowering to 33 or 22KHz, since the audio loses too quality), but I also chose to lower from stereo to mono and from the original high bitrate to 32Kbps. Very acceptable quality (for my needs) and faster encoding.

Finally, thank you for clarifying me the bitrate concept (at first I didn't realized the obvious fact that it was the cause of "unexplainably" having output files with lower framerates and framesizes but with the same file size, but later I realized that I didn't focus my mind of that so-simply reason: "why did I think about it earlier?").
But I'm not crazy: after all my tests, output movies with and without audio, at many different settings, I chose to encode my original movies at "160x120px 25Kbps 5fps audioMP3-32KBps48KHzMono" (and maybe I'll consider lowering from 48KHz to 44KHz for the same reasones explained above). For my current needs and devices/money availability, I prefer a unfluent 5fps movie with clear pictures and much faster encoded instead of a horrible fluent 25fps movie full of visual blocks and artifacts that takes much more time to be encoded. Even if these movies were to be watched for the first time and were not for temporarilty archived as is my need.

Currently I have a really basic encoding knowledge and that's the best I could obtain. If you have other smarter suggestions (other than buying a faster new and up-to-date laptop and other storage devices for which unfortunately I can't spend money at this time), I'll be very grateful.

Thank you to each of you for your attention and replies!

PS: please forgive me for my not perfect English, I'm not an American or English native speaker.
PPS: I hope not to have done concept mistakes in my text. I haven't got time now to re-read this post.
PPPS: I still don't understand why XVID4PSP encodes a file in twice the time the Handbrake takes, when I'm quite sure that the main video and audio settings are the same...

[jagabo]

The tradeoff of frame rate vs bitrate isn't linear. So 1/5 as many frames doesn't mean you can get away with 1/5 the bitrate. For example, I just encoded a 25 fps video at 160x120 at 25 fps, 12.5 fps, and 5 fps using x264 at CRF18. I got the following file sizes (no audio):

25 fps: 11.8 MB
12.5 fps: 9.9 MB
5 fps: 7.9 MB

I suggest you try 12.5 or 10 fps as that will still look fairly fluid, increase your encoding speed, and reduce the bitrate requirement.

[falco2000]

The tradeoff of frame rate vs bitrate isn't linear. [...]
I suggest you try 12.5 or 10 fps as that will still look fairly fluid, increase your encoding speed, and reduce the bitrate requirement.

Thank you jagabo. I finally decided to use 10 fps instead of 5 but I had to increase from 25 Kbps to 30 Kbps to reduce the consequent loss of picture quality (blocks and unfocused faces again ). This increased a little the final filesize, but maybe it's worth the more fluidity.

Thanks

[jagabo]

Why bother with bitrate at all? Use CRF (constant quality) encoding. The video will get exactly the right bitrate necessary for the quality you ask for. What difference does it make if the final average bitrate turns out at 15 kbps, 30 kbps, or 45 kbps?

[falco2000]

Why bother with bitrate at all? Use CRF (constant quality) encoding. The video will get exactly the right bitrate necessary for the quality you ask for. What difference does it make if the final average bitrate turns out at 15 kbps, 30 kbps, or 45 kbps?

It makes the difference that the filesize increases. I should do preview outputs with different quality values and check the final bitrate to decide the right quality value and a compromise with the final filesize.

[jagabo]

It makes the difference that the filesize increases.

So what? You've already admitted that increasing the bitrate from 25 to 30 was acceptable. And with some videos the file size may go down rather than up. You may even find that the total size of all your files is lower with a CRF values that's acceptable to you. Why is it so important to you that they all be the same size?

[falco2000]

It makes the difference that the filesize increases.

Why is it so important to you that they all be the same size?

You are probably right, but you said "What difference does it make if the final average bitrate turns out at 15 kbps, 30 kbps, or 45 kbps?". If for the first file I choose RF:30 for quality and the output has an average bitrate of 15kbps, the next one could have 45 kbps. I should check all the files I create.

By the way, what does RF means?

And what does "quality value" mean in the encoding program? The program can't watch the output movie with human eyes and decide if the picture quality of two movies is the same... So what does setting a fixed quality value for all movies means? (by the way, this is also true for setting a fixed bitrate )

[jagabo]

And what "quality value" means in the encoding program? The program can't watch the output movie with human eyes and decide if the picture quality of two movies is the same... So what does setting a fixed quality value for all movies means? (by the way, this is also true for setting a fixed bitrate )

Quality based encoding certainly delivers more predictable image quality than bitrate based encoding. Consider JPEG encoding where you select a quality level before saving. Every image saved at a particular setting retains a predictable amount of detail even though the size of the file may differ greatly. It's the same with video encoding.

Bitrate and quality based encoding are two sides of the same coin. With bitrate based encoding you pick the bitrate (and hence file size) and the encoder delivers whatever quality it can for that bitrate. With quality based encoding you pick the quality and the encoder uses whatever bitrate is necessary to deliver that quality. When the file size matches (eg, if you perform a quality based encoding and then go back and perform a bitrate based encoding at the same average bitrate) the results will be nearly identical.

If you're primarily interested in image quality you should use quality based encoding. If you're primarily interested in file size (eg, you have to fit a particular video on a CD or DVD) you should use bitrate based encoding.

Good bitrate based encoding requires two passes because the encoder has to examine the entire video before it can allocate bits to individual frames in order to meet your requested average bitrate. Quality based encoding only takes one pass because the encoder just uses whatever bitrate is necessary to meet your quality requirement at each frame. And guess what -- the first pass of a 2-pass bitrate based encoding is a quality based encoding to determine the bitrate required by each frame!

In x264 QP mode is a quality based method but it's very much mathematically based. CRF mode is a quality based method that takes into account what's more visible to the human eye.

[DB83]

To, hopefully, expand on what jagabo wrote, and also to answer your earlier questions.

'Quality' is an expression of the compression of a video. If you have a video which is fully uncompressed then that can be expressed as perfect quality. Compression reduces quality since it throws away information from the video so even if you come back to that compressed video at a later date and re-encode it with less compression you can not recover what is already lost.

RF is, I believe, the ReFerence point for the compression. It is expressed as a range of numbers where 0 is the same as uncompressed and the higher the number goes the greater the compression.

So CRF = Constant compression by the ReFerence number. These numbers will vary according to the efficiency of the codec you are using.

[jagabo]

In x264 RF stands for Rate Factor: http://mewiki.project357.com/wiki/X264_Settings#crf

I believe it's a starting quantizer value as in QP encoding: http://mewiki.project357.com/wiki/X264_Settings#qp

Then the encoder uses a higher or lower quantizer depending on what it determines the frame actually needs.