[Help Request] Sony Vegas 10 Pro, AVC/H.264 Encoding

[lindenkron]

Greetings,

First off let me apologise if this thread is in the wrong section. I didn't really know where to go and this is quite a large forum - any moderator feel free to move it to what ever section you feel would be appropriate.

I spent the last few weeks, a few hours a day, trying to master the craft of encoding videos at a low file size and good quality - probably the aim of most people that are making / rendering movies. However, I find myself in quite a desperate situation since I cannot seem to get beyond the point which I'm at now and therefore I humbly request aid of any kind that anyone on this site can supply for me.

My aim is to get the sharpness and quality equal or similar to this video:


To the best of my knowledge he's using H.264 (AVC) while rendering it in .f4v? (Not sure about the .f4v bit as I've not been able to dig up any information about this). There are countless amount of videos on youtube that show how to render in oh so amazing quality and it all seems fine and dandy when they do it, yet I've yet to replicate anywhere near the same result. I believe he uses Adobe Media Encoder, but it requires CS4 and I'm only in possession of CS3. Also my camera delivers .mts files which are not recognized by CS3 (only CS4, typical isn't it?) so I'm pretty sure this isn't a viable option for me.

I've tried:
.wmv (obviously a bad result, but low file size)
.avi (both compressed and uncompressed using Easy h.264 afterwards)
.mp4 (AVC - both the MainConcept AVC/AAC and the Sony AVC)
.mov

A little example of the original footage before touching anything. This clip is 45mb in size:
http://i56.tinypic.com/348lo42.jpg
(Note: It's not my on the picture)

After encoding it in .mp4 AVC (h.264) this is how it comes out, and it matters now if I use any effects to sharpen it up or what not.
http://i52.tinypic.com/2w5w1gn.jpg
(Note: It's not my on the picture)

Specifically look at the eyes and the shed in the background and you can easily see that there's serious sharpness issues etc.

These are my render settings for MainConcept AVC/AAC at the moment. I'm open for any suggestions as what I could try and do to improve my situation.


In advance a heartfelt thanks,
Lindenkron

[budwzr]

Use the Sony AVC encoder, MC one is no good. Use the 720p30 10K template, and you'll be fine.

[lindenkron]

Use the Sony AVC encoder, MC one is no good. Use the 720p30 10K template, and you'll be fine.

As mentioned in my original post I already tried the built-in Sony AVC encoder yielding the exact same quality as any other "professional" compression.

These settings were used in the following example:


You can clearly see the quality difference in the original and the compressed footage.


Any other suggestions?
Thanks!

[poisondeathray]

What format is your original footage? use mediainfo if you don't know (view =>text)

What project settings did you use ?

What editing have you done ? Any speed changes ? deinterlacing ?

How are you taking the screenshots ? What software?

You probably wouldn't use a 720p30 preset suggested above if you have 25p footage... By default, vegas will insert blurry blended frames for speed changes .

[lindenkron]

What format is your original footage? use mediainfo if you don't know (view =>text)

What project settings did you use ?

What editing have you done ? Any speed changes ? deinterlacing ?

How are you taking the screenshots ?

You probably wouldn't use a 720p30 preset suggested above if you have 25p footage...

As mentioned in previous posts, the original footage is .MTS, I believe it's a standard for video recording? Not sure. It's full HD 1920x1080, so even in 720p it should be good.

I usually run 25p as well, but previous poster wanted an example of using 30p - Same result though.

My project settings are:


I have done no editing to the video, no speed changes and no deinterlacing. I've not touched it for the sake of accurate results.

The screenshots are screendumps (Ptr Scr) of the final rendered movie in full screen mode. There's no difference between what I see in the movies and the quality of the pictures. I use Adobe Photoshop CS3 .JPG 12(max) quality. The quality loss at best is minor and unnoticeable.

Thanks to everyone taking their time to read my thread,
Cheers!

[budwzr]

You probably wouldn't use a 720p30 preset suggested above if you have 25p footage.

I thought the OP wanted it for YT. Why use 25p when 30p has more frames? Since OP thinks YT looks good.......

Hahaha, oh, I get it, the original footage is 25 frames...hahaha.

[poisondeathray]

not enough information

MTS is a transport stream . It's a container that can hold different types of video, audio, metadata. It doesn't say anything about the video

We need to know specifics. Is it interlaced ? What compression ? h264 ? MPEG2 ? What frame rate ?

mediainfo (view=>text) copy & paste the results back here


Normally you would use the same project settings as your source material. If 1920x1080, then use 1920x1080

[lindenkron]

not enough information

MTS is a transport stream . It's a container that can hold different types of video, audio, metadata. It doesn't say anything about the video

We need to know specifics. Is it interlaced ? What compression ? h264 ? MPEG2 ? What frame rate ?

mediainfo (view=>text) copy & paste the results back here


Normally you would use the same project settings as your source material. If 1920x1080, then use 1920x1080

I'm pretty sure if you scale downwards in the same scale it shouldn't affect the quality. I'll take a look at the link you send me if you'll give me a minute or two. In the mean while these are the file info window gives me:


Thanks!

Edit:

General
ID                               : 1 (0x1)
Complete name                    : C:\****\00000.MTS
Format                           : BDAV
Format/Info                      : Blu-ray Video
File size                        : 45.9 MiB
Duration                         : 22s 341ms
Overall bit rate                 : 17.2 Mbps
Maximum Overall bit rate         : 18.0 Mbps

Video
ID                               : 4113 (0x1011)
Menu ID                          : 1 (0x1)
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Format settings, GOP             : M=1, N=24
Codec ID                         : 27
Duration                         : 22s 0ms
Bit rate mode                    : Variable
Bit rate                         : 16.3 Mbps
Maximum bit rate                 : 16.8 Mbps
Width                            : 1 920 pixels
Height                           : 1 080 pixels
Display aspect ratio             : 16:9
Frame rate                       : 25.000 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Interlaced
Scan order                       : Top Field First
Bits/(Pixel*Frame)               : 0.314
Stream size                      : 42.7 MiB (93%)

Audio
ID                               : 4352 (0x1100)
Menu ID                          : 1 (0x1)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Mode extension                   : CM (complete main)
Codec ID                         : 129
Duration                         : 22s 112ms
Bit rate mode                    : Constant
Bit rate                         : 256 Kbps
Channel(s)                       : 2 channels
Channel positions                : Front: L R
Sampling rate                    : 48.0 KHz
Bit depth                        : 16 bits
Compression mode                 : Lossy
Delay relative to video          : -80ms
Stream size                      : 691 KiB (1%)

Text
ID                               : 4608 (0x1200)
Menu ID                          : 1 (0x1)
Format                           : PGS
Codec ID                         : 144
Duration                         : 21s 536ms
Delay relative to video          : -80ms

I hope that's what you were looking for
Thanks!

[poisondeathray]

The screenshots aren't even of the same frame, but I'll assume it's representative of the entire video

This really looks like mismatched settings , or you are doing a very soft resize, or you are blend deinterlacing (you have blend deinterlacing selected in the project settings, but its not activated unless your footage is interalaced and you export progressive). Low quality from encoding settings or a bad encoder doesn't look like this. The picture is globally blurry. Encoding artifacts look very different and aren't distributed this evenly

When you exported uncompressed AVI earlier, how did it look ? If that was blurry as well, it rules out compression as the culprit and it's most likely your project settings

[poisondeathray]

Scan type : Interlaced
Scan order : Top Field First

That's your answer

You are blend deinterlacing

Vegas sucks for deinterlacing. The other option if you use vegas is interploate, but you will notice jaggy artifacts. But at least it won't be as blurry

Many people resort to other programs for deinterlacing and encoding, but try "interpolate" first in the deinterlace method for the project settings

[lindenkron]

The screenshots aren't even of the same frame, but I'll assume it's representative of the entire video

This really looks like mismatched settings , or you are doing a very soft resize, or you are blend deinterlacing (you have blend deinterlacing selected in the project settings, but its not activated unless your footage is interalaced and you export progressive). Low quality from encoding settings or a bad encoder doesn't look like this. The picture is globally blurry. Encoding artifacts look very different and aren't distributed this evenly

When you exported uncompressed, how did it look ? If that was blurry as well, it rules out compression as the culprit and it's most likely your project settings

Yes, this is representative of the entire footage, I just picked out a random "close" footage of each of the rendered videos since it was the same all the way through.

I agree, it doesn't look too good which is why I'm here - I did export it once in .avi uncompressed which resulted in a 500mb file size which I then ran through Easy h.264 to compress it. This resulted in vertical lines running across the movie (interlaced I believe?) which was not really a result I was aiming for.

I'm a rookie at this rendering game - but I really need to figure out how to get this quality on a roll. I appreciate you taking your time with me, thanks again.

-lindenkron

[lindenkron]

Scan type : Interlaced
Scan order : Top Field First

That's your answer

You are blend deinterlacing

Vegas sucks for deinterlacing. The other option if you use vegas is interploate, but you will notice jaggy artifacts. But at least it won't be as blurry

Many people resort to other programs for deinterlacing and encoding, but try "interpolate" first in the deinterlace method for the project settings

I'm blend deinterlacing? - I turned off "Reduce interlace flicker" on the footage, and "Smart resample" is also disabled. Is there no way to get the quality desired without having to get Adobe CS4+ and Adobe Media Encoder? - Which is what I assume the guy in my before mentioned video was using. Some sort of .flv/.f4v file encoded with h.264.

[poisondeathray]

Try interpolate first, if you really want to learn a better method, you will have to learn how to use avisynth deinterlacers (QTGMC), and use a better encoder (x264)

But keep in mind, Youtube butchers everything. It might not be worth the effort if YT is your end goal

[lindenkron]

Try interpolate first, if you really want to learn a better method, you will have to learn how to use avisynth deinterlacers (QTGMC), and use a better encoder (x264)

But keep in mind, Youtube butchers everything. It might not be worth the effort if YT is your end goal

Well, from the videos I see on youtube it must be possible.

I will try and read up on these terms "Interpolate", "avisynth deinterlacers", "x264" and then return to this thread.

Cheers mate!

[poisondeathray]

I'm blend deinterlacing? - I turned off "Reduce interlace flicker" on the footage, and "Smart resample" is also disabled. Is there no way to get the quality desired without having to get Adobe CS4+ and Adobe Media Encoder? - Which is what I assume the guy in my before mentioned video was using. Some sort of .flv/.f4v file encoded with h.264.

yes you are blend deinterlacing. Why do you think it's blurry ? Blend= blurry

Blend means it combines 2 adjacent fields into 1 image

CS4 has a horrible deinterlacer and bad encoding quality too, I wouldn't go there . I've put up many comparisons before if you search....

[poisondeathray]

Try interpolate first, if you really want to learn a better method, you will have to learn how to use avisynth deinterlacers (QTGMC), and use a better encoder (x264)

But keep in mind, Youtube butchers everything. It might not be worth the effort if YT is your end goal

Well, from the videos I see on youtube it must be possible.

Not necessarily, the content is different. You are comparing apples and oranges

This is probably too much for you to begin with , but there are differences in the way compression works for different types of content. I won't go into big details, but certain types of content compress better than others. An interview piece like that example video will be much easier to compress than a video of a guy walking in your backyard with trees and grass. Most compression types use Long GOP comrpession - they encode the differences between frames. So things like motion or explosions, noisy footage, where there a difference between frames will require lots more bitrate to encode. Hand held footage will take lots more than stabilized, or tripod shot. Because YT limits the bitrate, this will destroy the backyard shot , but the static piece will look passable in quality.

So to optimize video for youtube, people stablize their shots, denoise their footage, intercut with static scenes. You can look up more details in other threads for how to optimize for youtube

[lindenkron]

Try interpolate first, if you really want to learn a better method, you will have to learn how to use avisynth deinterlacers (QTGMC), and use a better encoder (x264)

But keep in mind, Youtube butchers everything. It might not be worth the effort if YT is your end goal

Well, from the videos I see on youtube it must be possible.

Not necessarily, the content is different. You are comparing apples and oranges

This is probably too much for you to begin with , but there are differences in the way compression works for different types of content. I won't go into big details, but certain types of content compress better than others. An interview piece like that example video will be much easier to compress than a video of a guy walking in your backyard with trees and grass. Most compression types use Long GOP comrpession - they encode the differences between frames. So things like motion or explosions, noisy footage, where there a difference between frames will require lots more bitrate to encode. Hand held footage will take lots more than stabilized, or tripod shot. Because YT limits the bitrate, this will destroy the backyard shot , but the static piece will look passable in quality.

So to optimize video for youtube, people stablize their shots, denoise their footage, intercut with static scenes. You can look up more details in other threads for how to optimize for youtube

I get where you're coming from - There's just not really any place on the web that take things and boil em down like this. It's always 10 pages of technical details that I'm quite frankly not interested in spending my time reading about.

Basically what you're saying is that pictures with action in them, moving, takes more bit rate (information) to encode because more pixels move about hence requiring more data/info and since YT limits that, the quality will be shit in the end anyway? Right? If not I misunderstood and I apologise.

So basically - Lighting and steady shooting is a vital part of getting below a bit rate that YouTube accepts, right? - Would you happen to know what this rate is - would be nice to stay under it .

I did one movie, a music video, which was in .wmv and the quality turned out alright - I don't know what has happened since then but I can't seem to get anywhere near acceptable sharpness etc. anymore. Which is why I started looking into all of this. If you have any suggestions, clever ideas or wisdom you can pass on I'd greatly appreciate it.

I reallllyyyy need to make this work
Thanks, once again!

[budwzr]

How can 25fps be interlaced?

MeThinks the OP is a dinkerer, and therefore nothing is as it seems. Hence the confusion.

[lindenkron]

How can 25fps be interlaced?

MeThinks the OP is a dinkerer, and therefore nothing is as it seems. Hence the confusion.

1. I have no clue.
2. What is a dinkerer?
3. What's the confusion if I may ask?

Thanks.

[budwzr]

Someone who "dinks" with all the settings and creates a type-mismatch somewhere in a system or process, then presents the problem in a generalized way as if all was normal up to that point.

[lindenkron]

Someone who "dinks" with all the settings and creates a type-mismatch somewhere in a system or process, then presents the problem in a generalized way as if all was normal up to that point.

Considering I just installed Vegas prior to doing this, I don't see what all these settings are that I supposedly "dinked" (I don't believe that's actually a word mate) with.

[edDV]

There are several separate processes going on that you don't seem to grasp. All contribute to end quality. I'll attempt to overview them.

1. Camera source - You have given no camera details but it appears to be 1920x1080 25i AVCHD.

2. Shooting conditions - Lighting, camera stabilization and subject motion are critical to ultimate encode quality. In Film class they would call this "shooting for compression".

3. Editor project settings - As said above, in the typical case you would match the camera format. Current CS5 and Vegas 10 versions have 1920x1080 25i AVCHD templates and h.264/AC3 codec support. CS3 does not so you would set project to 1920x1080 25i upper field first and hope for the best.

4. Deinterlace - You are shooting interlace video and Youtube displays progressive. Deinterlace contributes heavily to the end quality. Experts use different methods for different types of video. Vegas allows three (no deinterlace, blend, interpolate). Selection is made in project settings. If there is any motion beyond a talking head, use interpolate. As said both Premiere and Vegas (also Final Cut) have mediocre deinterlace quality.

5. Frame rate - frame rate conversion will cause interpolation blurring similar to deinterlace. Better keep it 25 fps and let Youtube handle it since they are going to re-encode to their requirements anyway. Others may have opinions on 25 fps strategy for Youtube.

6. Resize - Above you resized to 1280x720. Resizing interlace video is a tricky optimization. Again Premiere and Vegas do a deinterlace followed by frame size interpolation. This gives a so so result. Experts would deinterlace and resize in other programs.

7. Encoding - This gets tricky because Youtube is going to re-encode your work in most cases. Premiere and Vegas control deinterlace, frame rate, frame size and encoding all in the export encoding menu. The main encoding choices will be codec, interlace vs progressive and bit rate. Recent versions offer templates for internet upload. IMO and this is debatable, since Youtube is going to re-encode anyway, it is better to max the bit rate that you send them (limited by maximum file size allowed) to get the best re-encode at Youtube.

Those are the big picture issues. Good luck.

[lindenkron]

1. I appreciate you taking the time to write that up for me edDV, I really do. I'm most likely getting the Canon 550D (T2i) in two weeks and then the i/p difference won't be a problem. I honestly didn't realise that the camera (I'm borrowing one) I have right now was shooting in 1080i and not 1080p. And the reason I've given no camera details, is because, as mentioned, it's not my camera/video recorder so I hardly know anything about it .

3. There's no standard for 25i AVCHD in the templates on Vegas Pro 10.0.


I checked the settings changed when picking 1920x1080-60i, 10 Mbps video stream, and it appeared that the only thing that changed was the "Field order:" that changed to "Upper field first"

So I went ahead and made my own settings and changed that.


Which resulted in me receiving this message.


4. I changed the Deinterlace method to Interpolate fields, but I've yet to be able to try it out due to that error.

6. What other programs would they resize in?

7. That seems like a smart way to see it, but I still need to manage to optimize the picture so it looks decent - even if not caring about file size to some extend - any suggestions?

Thanks!

[poisondeathray]

3. There's no standard for 25i AVCHD in the templates on Vegas Pro 10.0.

25i and 50i denote the same thing , just different naming conventions. They both mean 50 fields per second or 25 frames per second

Use the 1080-50i preset

I checked the settings changed when picking 1920x1080-60i, 10 Mbps video stream, and it appeared that the only thing that changed was the "Field order:" that changed to "Upper field first"

Don't use the 60i preset. 50i=>60i speed change will introduce blended frames (blurry) when "smart resample" is on, and duplicate frames (choppy playback) when "smart resample" is off. When the settings match, everything works . So use the matched settings as your source.




Not sure about the other errors, but try using the proper project settings first.

[lindenkron]

Well that had an interesting effect...
Using 1080-50i and then start rendering crashed my Sony Vegas on 0% (I managed to see the render bar). Hmm.

[poisondeathray]

Reboot your system, reload the project, and try the mainconcept AVC export and see if it makes a difference

Is your computer stable ? Is it overclocked ? Have you check your memory integrity ? System temps ?

Quite a few people on various forums seem to be finding vegas 10 unstable.

If your goal was youtube, you should not be selecting upper field first in the export settings (you want to select progressive, so vegas will apply a deinterlace) . You should use CABAC instead of CAVLC (higher quality and compression) , profile high instead of baseline as well (baseline imposes more limitations on what features a h264 encoder can use, so you end up with lower quality at a given bitrate)

[edDV]

Vegas Pro 10 project setting for AVCHD

[lindenkron]

Vegas Pro 10 project setting for AVCHD

Those were the settings I used yes

Reboot your system, reload the project, and try the mainconcept AVC export and see if it makes a difference

Is your computer stable ? Is it overclocked ? Have you check your memory integrity ? System temps ?

Quite a few people on various forums seem to be finding vegas 10 unstable.

If your goal was youtube, you should not be selecting upper field first in the export settings (you want to select progressive, so vegas will apply a deinterlace) . You should use CABAC instead of CAVLC (higher quality and compression) , profile high instead of baseline as well (baseline imposes more limitations on what features a h264 encoder can use, so you end up with lower quality at a given bitrate)

I'll try rebooting - It's not really a "Project" it's just the same movie clip I use for testing. Just so I know it's not a VideoFX flaw or what not. Once the quality is how I want it, I'll advance from there - just to avoid complications later on.
My computer is new, it's overclocked from the manufacturer, I've done no handyman job on it myself. How do one go about checking memory integrity and system temps? Don't know if you have seen the specifications for my rig, but I believe it should be able to handle this :P. i7 core, 3.4ghz, 8gb ddr3 1600 mhz ram - should be plenty.

It would seem that Vegas is a tad unstable ye, but it must be something conflicting because it only does it on specific things. Such as trying to pull off a 50i render. I'll make sure the things set are set and try again after reboot .

Thanks again both, really appreciate it.
Brb 2 mins!

[poisondeathray]

My computer is new, it's overclocked from the manufacturer, I've done no handyman job on it myself. How do one go about checking memory integrity and system temps? Don't know if you have seen the specifications for my rig, but I believe it should be able to handle this :P. i7 core, 3.4ghz, 8gb ddr3 1600 mhz ram - should be plenty.

that's a very mild overclock and should not be any problem, but you can still have hardware issues, like a stuck fan, or corrupt memory

you can check your temps with something like coretemp or realtemp. It's important to check loaded temperatures, not idle. So use a program like linx, prime95, occt, intel burn test, etc.. there are many others... while observing temps

run memtest86+ on a dos boot disk to check your memory

[lindenkron]

My computer is new, it's overclocked from the manufacturer, I've done no handyman job on it myself. How do one go about checking memory integrity and system temps? Don't know if you have seen the specifications for my rig, but I believe it should be able to handle this :P. i7 core, 3.4ghz, 8gb ddr3 1600 mhz ram - should be plenty.

that's a very mild overclock and should not be any problem, but you can still have hardware issues, like a stuck fan, or corrupt memory

you can check your temps with something like coretemp or realtemp. It's important to check loaded temperatures, not idle. So use a program like linx, prime95, occt, intel burn test, etc.. there are many others... while observing temps

run memtest86+ on a dos boot disk to check your memory

Ah, if you're talking chip temperatures I have checked that once. It was 6 degrees higher than what was recommended if I remember correctly. I have had some frightening experience with this rig though while rendering.

Normally the CPU doesn't go above 60% while rendering, but while rendering .mp4 a few times it have put all 8 cores at 99%, which normally would be a good thing. Only thing is this time the CPU/Power Supply/Or something else started making a really frightening high pitch noise that sounded like it was on the edge of breaking, like... major frightening... and I have no idea what it was... I got it recoded if you want to hear it. I'm not sure it's related to this topic though.

Back on topic:
Original footage 45.8MB
Footage after rendering .mp4 27MB

So the file size have gone down a bit, but so have quality (not by a whole lot though! ). Here's the aftermaths:


It would appear rebooting did the trick hehe.
So where do I go from here, I need to get it down to 720p since 1080p is excessive and unnecessary.

Also; Would you recommend using a VideoFX "Sharpen", just the basic one, to try and sharp things up?

Thanks a lot both of you!