Help me reduce these huge video files! How does the "scene" do it

[AnitaPeterson]

Hello everyone,

I'm a happy owner of a Canon SX1 camera, which replaces my 3 year-old Canon S3.

However, this camera produces HUGE 1080p video files in MOV format.

By HUGE, I mean that a single 3-minute clip is somewhere in the vicinity of 1.3 GB, this is ridiculous!!!

Over the last three months, since acquiring the camera, I attempted to use various applications to reduce the size of the video... the best result, in terms of size, was 400 MB for that same 3-minute clip, using Vegas.

I'm not completely Internet-challenged, so I know about "the scene"... How do these guys succeed in getting such great qualities at a much-reduced size?

Take for instance SecretMyth (Kingdom-Release)/Kingdom RG.... if you know who they are, you know how good their encodes are. They achieve a lot of compression with a minimum of image quality loss.

How do they do it?

I wish I could encode my 1080p camera files with whatever it is they're using. These guys pack 2 hours in 720p quality, at 120 minutes in length, using a mere 2 GB for the whole thing.

I'll be damned if I can get such superb compression using Ulead/Corel VideoStudio, Vegas, Pinnacle or other pieces of software I tried over the last few months...

Anyone?

[neomaine]

Nice camera! I looked at the SX10 and the SX1 but ended up going with the Pany DMC-FZ35. About $150+ less than the SX1/SX10 but also has only 720p video but useable zoom during video. But what I'm really jealous about is the flip LCD... though, I'd probably just break it.

Anyway, welcome to 1080p video! First your comparison isn't straight on. At 720p (1280x720) they have 1/4 (half the vertical and half the horizontal) resolution to work with compared to your 1080p (1920x1080). Given the same length of video with the lower resolution they can also afford 1/4 the bitrate which means... 1/4 the filesize. Filesize = duration * bitrate. Nothing else matters in the equation. These guys are also copying bray movie so they're already working with much cleaner video, not the messy movement/snowy/grainier (relatively) that you are.

For your numbers, to figure out bitrate (bitrate = filesize/dur), so 1300Mbytes/180secs = 7.2Mbytes/sec. Sounds about right, and at the low end for 1080p movies. You can use gspot or media info on your MOV file to get actual numbers.

I have no idea who these guys are but from what you've described they probably use vbr h264 for video and probably use vbr mp3 to keep the audio bitrate down. MKV seems popular for a container these days but I've only dabbled with it up until now. You should go it that direction. I've backed up most of my DVD movies using divx CQ =2 and kept AC3 audio. Usually get my movies to 1.5 to 2.2gb per movie (some lower, some higher depending on the content). I'm more concerned about quality than size so I use CQ.

Just changing you resolution to 720p you could drop your bitrate to 1.8Mb/sec without (theoretically) changing the quality. Watched on a 720p or less television, you probably wouldn't know the difference.

Now, how do you get it there? Personally, I’d try finding a vwf MOV codec and use VirtualDub out with x264 (I think you can do that). Or, some tools like AutoGK, Xvid4PSP, Handbrake… finding settings that use h264.

I’d go more into how to use those tools but its better for you to futz around, figure things out and ask your encoding questions afterwards.

NOTE: Hard drives are cheap these days. I'd recommend keeping as much original footage as you can so you can always go back if needed.

PS: I hope my math was right...

[Gavino]

At 720p (1280x720) they have 1/4 (half the vertical and half the horizontal) resolution to work with compared to your 1080p (1920x1080).
...
PS: I hope my math was right...

1280x720 is 2/3 the vertical and horizontal resolution of 1920x1080, so overall its 4/9 (44.4%), not 1/4.

[AnitaPeterson]

Hmmm...

As you can, maybe, see from my post, I've only used commercial packages so far.. Someone suggested I should use x264, but that's a command line application, and I'm not familiar at all with these...

Can anyone suggest me where - and how - to begin this?

[poisondeathray]

x264 has many easy to use GUI's e.g. ripbot264, xvid4psp, handbrake , which have presets, and generate scripts for you. You can start with those. If you want more advanced settings: http://mewiki.project357.com/wiki/X264_Settings

The other part of the equation is filtering: e.g. stabilization of footage, denoising noisy footage, both which can substantially reduce bitrate requirements for a certain "quality level"

[deadrats]

Take for instance SecretMyth (Kingdom-Release)/Kingdom RG.... if you know who they are, you know how good their encodes are. They achieve a lot of compression with a minimum of image quality loss.

How do they do it?

I wish I could encode my 1080p camera files with whatever it is they're using. These guys pack 2 hours in 720p quality, at 120 minutes in length, using a mere 2 GB for the whole thing.

sure i know of them and i know of their releases and one or two other highly regarded "scene" groups and personally i think their work is highly over-rated.

it takes a stretch of the imagination, not to mention poor eyesight, to come to the conclusion that the files the release are anywhere near the quality of the blu-rays they use as source. the originals are 1920x1080p, using either h264 or mpeg-2 at bit rates greater than 25 mb/s, with lcpm or 640kps ac3 audio and they shrink that to 1280x720p, less than 3mb/s x264, 320kps ac3 and wrap everything in a mkv container.

any clown can do that: use any x264 front end you want (or do it from the command line, if you choose), set the target bitrate at 3 mb/s, 2 pass, apply a re-size filter first, re-encode the audio to a lower bit rate and viola, you have taken a nice high quality source and produced a crappy 1/10 scale (in terms of file size) duplicate.

now for the reality check: your camera is designed to record the highest quality image possible at it's listed specifications, basically it's capable of taking pictures in RAW or jpeg format, at 10 mega pixels and the full hd mode is recording at 1920x1080 dv format (that's why it's using the mov container), from what i can tell it looks like the dv compression it uses for video is in the 25mb/s range, the same as a commercial blu-ray disk, no wonder the files are "huge", according to online reviews you can shoot a maximum of 4gigs or 1 hour worth of video, which ever comes first, and in full hd mode the 4 gig limit is reached in about 12 minutes.

in all honesty any good encoder should allow you to re-encode to 720p at 5 mb/s, which would reduce the size of the file to 1/5 the source, but i would never target the file sizes that the "scene" groups use as a goal.

but that's just me...

[edDV]

The SX1 shoots to h.264 intended as a final distribution format.

If you recode h.264, you will suffer loss.

If you want smaller files, check camera settings for 1290x720p or less. 60fps is better than 30fps but the file size will be double.

If you want the best for web, shoot uncompressed 1080i/p or 720p with a pro cam, edit and compress from there.

But I must ask...

If you don't want HD why did you buy HD? You spent $600 for a logo?

[AnitaPeterson]

The SX1 shoots to h.264 intended as a final distribution format.

If you recode h.264, you will suffer loss.

If you want smaller files, check camera settings for 1290x720p or less. 60fps is better than 30fps but the file size will be double.

If you want the best for web, shoot uncompressed 1080i/p or 720p with a pro cam, edit and compress from there.

But I must ask...

If you don't want HD why did you buy HD? You spent $600 for a logo?

Thank you for asking! The tone may be a bit harsh, but the question is legit.
I already had a Canon TX1, which did .AVI files in 720p, which were indeed manageable. After the purchase of the SX1,
I found the newly-adopted .MOV format to be a headache. It's hard to edit, and requires specific software, besides sizable hardware. I am waiting for these to come down in price, and I'm wondering about storage options in the meanwhile.

I already got a Blu-Ray burner, but the disc creation software I see is rather poor, and blanks are still expensive. Therefore, I am willing to part with some quality, in return for a sizable gain of effective storage space. Bottom line: These will not be professional productions, they're personal videos - which I'd like to have access to at the best possible quality/size ratio, while still above a "regular DVD" quality.

As far as the SX1 goes, I really like it as a hybrid camera - quite capable in both photo and video terms; it was a replacement for a much-loved-and used S3. But I'm afraid it's a bit ahead of its time, in terms of HDD space.

[fourCCrider]

hi AnitaPeterson!
if you're into editing, may i point you to Eugenia Loli's blog entry on editing giant canon .mov files - links to her tutorials included, in which she recommends proxy editing in Vegas (or buying Cineform Neoscene...)

http://eugenia.gnomefiles.org/2009/09/16/editing-canon-5d7d-footage-on-windows/

however, every recoding process leads to loss of quality - haven't seen any exceptions so far. i once reduced Full HD blu ray m2ts to mkv for a WDTV box with handbrake 0.93 (~20 GB to ~3 GB), but you can easily tell the difference even on an old analog tv - so don't squeeze too much...

[midnightsun]

For your numbers, to figure out bitrate (bitrate = filesize/dur), so 1300Mbytes/180secs = 7.2Mbytes/sec. Sounds about right, and at the low end for 1080p movies.

You're mixing bytes with bits, 7.2MBytes/s is equal to 7.2*8=57.6Mbits/s, which isn't "low end" for 1080p

[neomaine]

Ah, much better math, and it just makes more sense too... Thanks for the correction!