my question is simple, whether using the CQ scale or 2-pass scale ripbot does not lock bitrates. for my encodes from blu-ray, i've chosen 2-pass @ 4096 kbps. i can play my encoded movies on my PS3 (profile used is HD Progressive with 1920x1080 res), the PS3 has a feature that allows me to see the bitrates dynamically as the movies play. in almost every movie i've played on the PS3, i've noticed bitrates that were variable, anywhere from 1 Mbps to 12 Mpbs.
can anyone clarify for me why the codec is varying the bitrate when i have a certain value set ? i do not lock the file size, so when i set the bitrate i would have expected that it remains around the neighborhood of what i set it. instead i get some points that are really low and really high.
+ Reply to Thread
Results 1 to 30 of 43
You're using 2-pass variable bitrate encoding. You select the average bitrate. The encoder uses more for scenes that need it, less for scenes that don't.
i didn't know that it was 'variable'. but isn't it also variable with CQ mode as well ? with 2-pass i noted that the bitrate often goes below the value i set, which i don't really want happening. but even at low bitrates the picture rendered still looks good.
i was thinking about moving on to a higher bitrate for 2-pass, something like 6144 ... or also thinking about going lossless. is there a good/reliable video codec that i can use to create a compressed lossless blu-ray movie with ? and would i be able to use a container like mkv for a lossless compressed movie ?
Constant bitrate should only be used when you have a player that doesn't support variable bitrate. For example, some old DVD players could only play CBR Divx files.
Use bitrate based encoding when you want a file of a particular size but don't necessarily care what the quality is. Use CQ encoding when you want a file of a guaranteed quality but don't care much what the file size turns out to be.
Last edited by jagabo; 12th Sep 2011 at 21:29.
ripbot goes about doing it's business, so i had to guess at what these settings do ...
concerning CQ, i would maybe switch to it if i knew what the scales focused on, as in what target bitrates would i be looking at for the various CQ settings. i also don't know how file size will be affected since i haven't tried using that method yet to yield an mp4 from a blu-ray.
my current 2-pass/4096 kbps method (and 320 kbps audio) gives me a consistent size factor, i can figure out the size of the mp4 just by knowing how many minutes the runtime is: for all my movies done in this method, if i multiply the # of minutes by 0.031GB, that will give me the final mp4 size.
i am not insanely conscious of output file size, but i'd like to keep it reasonable, like below 7GB for 3 hours runtime. this is because i've got a few hundred movies on bd, and plan on putting them all on my network. if i had a petabyte array, i wouldn't care to encode at all and save myself thousands of hours doing these movies. but realistically speaking i am working with just a few TB to start, then plan to add more space as/if i reach my limit.
file size = bitrate * running time.
ripbot uses for CQ mode. The x264 codec has two contant quality modes, CRF and QP. QP is constant quality in a mathematical sense. CRF takes into account what's more visible to the human eye and reduces bitrate further without losing visible quality. I use CRF=18 for my DVD rips. That usually give a file between 1 and 2 GB, sometimes less, sometimes more. With CRF and QP smaller values give higher quality (and hence larger files), larger values give lower quality (and smaller files). You can use values from 0 to 51. Just encode a few short representative videos with different CRF values and see what you're happy with. I suspect you're going to be using something in the low 20's.
thanks again Jagabo ... seems you know a lot about ripbot.
can you give me an example of an mp4/mkv from blu-ray that was done using CQ 18 and CQ 20 (file sizes, average bitrate, etc.) ?
is there anything good to be said about the 2-pass method ? i assume the second pass refines the first pass, to better optimize the bitrate for the movie being encoded ....
something else i noticed (rare) is that the codec sometimes breaks down in a few scenes which are dark. i noticed in one of my movies there was heavy pixelation in a dark scene, the blacks were big chunk of black/dark grey blocks. it's only during a short period of the movie, few seconds at most. did some searching around and found a few others report this type of problem .... is this actually the codec or a setting that can be changed to improve/minimize this ?
ripbot at all. But I know a bit about h.264 encoding, x264 in particular.
Those are Xvid encodes, not h.264, and the examples are extreme, but the issue is the same. One video required 20 times more bitrate than the other to maintain similar quality (relative to the source). In the real world the differences aren't that extreme but it's very common to see a 2x or 3x difference.
ok, if the only difference between CQ and 2-pass is to permit file size limits, then i'd be best served switching over to CQ as it would take less time to encode the movies. just to be sure, the encoding process is the same, just the target bitrates would be slightly different, correct ? somehow though i feel like i actually like the 2-pass process, just because it grants me the ability to set the average bitrate with choices that indicate the bitrate, rather than choices that do not indicate a bitrate (CQ). perhaps there are more choices with h.264 but ripbot has only presented some of them ....
concerning the pixelation/blocky darks, you are right, i tried playing back one movie i know i saw this anomaly in, but this time thru my TV and the pixelation in the same exact scene was not there at all. so video-wise, the encoder didn't screw up. it was probably the video display settings on my laptop's screen, i'll have to play around with it some time and figure out where i can improve or eliminate the problems reproducing black/greys. it is strange because i never saw this type of thing before on the display prior to watching the encoded movie. including playback from the source. i use wmp to play back the movies i make, so maybe it could be that ....
thanks for all your input ... i really appreciate your help answering my question regarding encoding.
i will mess around some with clips and see where the settings take me.
currently the resolution of 1920x1080 and settings i chose play back fine on my computers and PS3, so i am comfortable with that ... ideally, i will move on towards maximum bitrates and quality settings .... it's the next best thing to having to pull a disc from my collection and play it when i want to watch something.
i do have one more thing to ask you, this being on the audio side of the encoder ...
as i've been using mp4 format, the only available audio formats are AAC 5.1 channel. how does the encoder resolve 6.1 and 7.1 audio ? if i want to maintain the extra channels for movies that have them, would mkv format be a better choice for those movies ? not sure if mkv would support all the available audio codecs used on blu-rays ... but a few candidates i can think of would be the star wars saga and lotr EE's
I don't know what ripbot will do with your 7.1 audio. I always keep the original audio (usually just the English track) and use an MKV container. Of course, I'm usually using DVDs as source so the audio is usually 5.1 AC3.
I happen to have one Blu-ray rip sitting on the computer right now. It's a 20 GB MKV file, about 25000 kbps h.264, a 2.35:1 movie so it has black borders top and bottom. I recompressed, no cropping, with x264 at CRF 18 and the resulting file was about 7000 kbps. I'm running a CRF 21 encode now. It's not complete but it looks like the video will be about 4000 kbps.
Last edited by jagabo; 14th Sep 2011 at 17:56.
bitrates for video blu-ray can peak fairly high, i've observed the average bitrate of some movies i've encoded around 18-25 Mbps (max peaks of around 35), so about 1/2 of that would be 9-12 Mbps. it looks like CRF 18 is producing an output that is targeted at this range ....
with 2-pass, the max bitrate i can choose is around 8 Mbps, if what we've discussed is correct, it can yeild a movie quality slightly higher than CRF 18 because the average bitrate will be higher ...
i think i will choose the 2-pass 8000 kbps method to do the movies i enjoy the most or will benefit the most from the highest bitrate possible with h.264 .... or i could just leave them as m2ts files ...
Remember, that's one particular movie (and it partly consists of black borders which compress very well). Different movies may vary by 2 fold or more.
You have to stop thinking in terms of bitrate being proportional to quality. For a particular movie the more bitrate the better the quality. But different movies will require different bitrates to maintain that amount of quality. You should try sample encodings and see what quality level you're happy with, then encode all your movies with that quality level*. It doesn't particularly matter what movie or clip you use as a test. A particular CRF value will give the same quality with all sources. A particular bitrate will give different quality with different sources.
Think of CRF and QP encoding as specifiying how much detail you want to throw away. The more detail you throw away the worse the video will look but the smaller the file will be.
Bitrate maps of the two encodings:
Note the vertical scales are a little different. By the way, those encodes took about 75 minutes each.
* Of course you don't have to encode all your movies at the same quality level. On some you may choose to use a lower quality because you don't care about the picture, on others you might use a higher quality level because you care more about the picture. But get a feel for what the range is.
Last edited by jagabo; 14th Sep 2011 at 19:35.
good info to have ...
some movies are more intense in data than others. with DVD, all movies are in MPEG2, but with blu-ray there are 3 codecs that may be used, i am guessing with MPEG4/AVC the amount of video data stored may be greater than with MPEG2 or VC-1, or it may just be more complex ....
so in the case of what you encoded above, it appears as though more data was thrown away with the crf 21 encode. but doesn't all this equate to bitrates somehow ? how else would the quality scale be guaged ? in the crf 21 encode, it looks like the max bitrate is capped (compared to the crf 18 encode), but even then the max bitrates are both in the range of HD bandwidth.
i don't have bitrate viewer, but mediainfo gives me this info for my longest running movie (sorry if it's a little hard to read, all the encoding settings are listed):
Format : MPEG-4 Format_Commercial_IfAny : XDCAM EX 25 Format profile : Base Media Codec ID : isom File size : 6.20 GiB Duration : 3h 21mn Overall bit rate : 4 416 Kbps Encoded date : UTC 2011-08-26 09:58:18 Tagged date : UTC 2011-08-26 09:58:18 Cover : Yes Video ID : 1 Format : AVC Format/Info : Advanced Video Codec Format_Commercial_IfAny : XDCAM EX 25 Format profile : High@L4.0 Format settings, CABAC : Yes Format settings, ReFrames : 3 frames Codec ID : avc1 Codec ID/Info : Advanced Video Coding Duration : 3h 21mn Bit rate mode : Variable Bit rate : 4 096 Kbps Maximum bit rate : 25.0 Mbps Width : 1 920 pixels Height : 1 080 pixels Display aspect ratio : 16:9 Frame rate mode : Constant Frame rate : 23.976 fps Color space : YUV Chroma subsampling : 4:2:0 Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 0.082 Stream size : 5.75 GiB (93%) Writing library : x264 core 114 r1924 08d04a4 Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=4096 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=25000 / vbv_bufsize=25000 / nal_hrd=vbr / ip_ratio=1.40 / aq=1:1.00 Encoded date : UTC 2011-08-26 09:58:18 Tagged date : UTC 2011-08-26 10:03:23 Audio ID : 2 Format : AAC Format/Info : Advanced Audio Codec Format profile : LC Codec ID : 40 Duration : 3h 21mn Bit rate mode : Variable Bit rate : 317 Kbps Maximum bit rate : 335 Kbps Channel(s) : 6 channels Channel positions : Front: L C R, Side: L R, LFE Sampling rate : 48.0 KHz Compression mode : Lossy Stream size : 456 MiB (7%) Title : Ä Language : English Encoded date : UTC 2011-08-26 10:02:14 Tagged date : UTC 2011-08-26 10:03:23
As a highly simplified example, say we have four grayscale (0-255 range) pixels:
100, 101, 110, 115
Say we use a "quantizer" of 1 -- meaning you can consider a change of 1 between consecutive pixels to be inconsequential. 100 and 101 are both the same by that measurement. So those four pixels can be represented by only three values, 25 percent compression:
100, 110, 115
Now say we used a quantizer of 5. By this measure 100 and 101 are the same, and so are 110 and 115. So we can reduce the original list of four pixels to two values, 50 percent compression:
If we use a quantizer of 20 we can reduce the list to one value, 75 percent compression:
Now lets start with a different four pixels 50, 100, 150, 200. Even with a quantizer of 20 we can't eliminate any of the values. So this different video did not compress at all with a quantizer of 20, whereas the first video compressed to 1/4 the original size.
i had a look at the link referencing the 3 videos but they are static images when i click them. regardless, i follow what you're saying. the CQ scale speaks in terms of % compression rather than bitrate, so bitrate is actually worked out as the byproduct of the data and size of the movie when a certain percentage of CQ is chosen. so it is possible that 2 different movies, both encoded with the same scale setting (CQ 18 for example) can result in different bands of average bitrates based on the information contained ... is this correct ? if so, is this scale fixed in percentages ? how does the encoder know where to go with this in terms of what average bitrates will be observed in the output file ? the encoder contains the equations, but the input is parameters of the input signal ... so there has to be some reference point to start with.
i may have glanced over DCT's in college briefly (i am a mechanical engineer) but don't remember them as most of my dealings in the professional world haven't yet dealt with them. seems as tho they are a driver for video/image rendering software and the equipment used (cameras, etc.) ... interesting stuff.
* except in the case of an encoder which lets you specify the minimum and/or maximum allowable bitrate. In those cases, if the bitrate falls outside the min/max range, it will use a different quantizer to keep the bitrate within the range.
Just my $.02 worth, but I just can't seem to get comfortable with CQ on my BR rips. Most recently, I ripped "Heat". Since I like to playback through a PS3, I had to convert the VC-1 video stream to AVC and I just couldn't seem to get the blocking to go away in dark scenes even at CQ=12. However, setting it to 2-pass at a destination size similar to the original (about 28GB) got them to dissappear. I've pretty much given up on using CQ because of this.
In my experience most people who complain about blocking in dark areas have their black level set too high. Those areas supposed to be nearly black and the blocks essentially invisible.
If you've adjusted your black levels and still feel a need to reduce blocking in dark areas poisondeathray had some suggestions for this recently:
I've tried comparing CRF and 2-pass encodings many times (ie, create CRF, make a 2-pass encoding with the same average bitrate) and haven't seen significant difference it dark areas. Or anywhere else for that matter. Do you have any examples?
Last edited by zero7404; 16th Sep 2011 at 10:54.
I'll take smitbret's word that his TV is calibrated correctly and that he can see posterization and blocking artifacts in dark shots. I see them too if I bother to look for them. But I see them with both CRF and 2-pass encoding. If a make a CRF encoding, then go back an make a 2-pass encoding at the same average bitrate, both files look pretty much the same.
or smitbret may have changed advanced settings associated with the video profile ... this might lead to block artifacts ... along the lines of what you mentioned previously with the 4 pixel example
if he can do a screenshot (printscreen) at the moment this is observed, then examine the image, would the pixelation be visible ? if that's the case, then it would be an artifact driven by the encoder settings. if the anomalies are not present in the still, then would it be safe to assume it is a video output issue on the pc/display end ?
So, inspired by this thread and my eternal hatred of shadow blocking, I re-ripped Heat and tried a re-encode with Ripbot264. I set the CQ to 18 and changed the b-frames setting from 4 to 0. Final size was a little over 11GB. Guess what, no blocking. Some of the skies and blank backgrounds are still a little bit busy but I think I only noticed because I was looking for it.
I'm re-running it now with CQ set to 16 to see if the extra bitrate clears it up and I'll report later.