Improving seek in x264 encodes

[Surlias]

I think keyint and min-keyint are the settings that affect seeking in x264 encodes, but I'm not sure how to properly manipulate them. Basically, I want to make my encodes as "seekable" as possible, because currently, XBMC sometimes chokes on jumping forward or back with my encodes and simply refuses unless I spam it which is really annoying. I don't mind if this causes the resulting files to be larger (within reason).

Any tips?

[Detmek]

Set max keyint to 4 or 5 seconds (fps * 4 or fps * 5) and see how that works. You don't need to change min keyint.

[Surlias]

I've been letting it use the default keyint=240 and keyint_min=24 values so far. Here's my full preset:

cabac=1 / ref=5 / deblock=1:-3:-3 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=0 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc=2pass / mbtree=0 / bitrate=12364 / ratetol=1.0 / qcomp=0.65 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

[ndjamena]

If there's particular points you'd like to seek to, using a qpfile and setting I-frames on those points is an option. X264 tends to put I-frames on scene changes anyway, but it doesn't work very well for black frames and fade-ins.

You could also try telling XBMC to only seek on I-frames, which should speed things up, especially when combined with a slightly lower keyint.

[sophisticles]

You could also adopt the Blu-Ray standard of setting keyint equal to the frame rate.

[Surlias]

So keyint=24 and keyint_min=24 ? Or would it be keyint=23.976 ?

[Detmek]

Keyint must be integer, so 24. Do not set minimum keyint, let x264 do that. Besides, min-keyint can not be higher then keyint/2+1.

But, if XBMC has that setting, set it to seek on keyframes only. If it doesn't have that setting, create a few test clips with different keyint values and try.

[jagabo]

Easiest seeking: keyint=1 but compression will go to crap.

Basically, the shorter the GOPs the easier the seeking and the worse the compression (lower quality at the same bitrate, or larger files with CRF encoding). But the difference isn't too bad until you get below 30 or so.

[Surlias]

XBMC uses FFMPEG for decoding, so if FFMPEG supports keyframe seeking then it should be possible to enable it in XBMC. However, I haven't been able to find a way to do it. No wiki or forum threads explaining how.

I don't suppose anyone more experienced can help me find the setting (if it exists)?

So far, there doesn't seem to be much quality difference between keyint=240 and 24 using my preset ~12 Mbps average bitrate.

[ndjamena]

Unlike VLC and MPC-MC it doesn't look like XBMC has that setting. I guess you're stuck re-encoding with keyint 24.

[jagabo]

Here's a rough example. A 10000 frame sequence from a Blu-ray M2TS file cropped to 1920x816 and encoded with x264 with the slow preset:

keyint  file size (MB)
----------------------
 4      191
 8      143
 16     118
 32     104
 64      97
128      92
256      91

Of course, the exact results will vary depending on the properties of the video. For example, music video with lots of very short shots will benefit less from long GOPs.

[poisondeathray]

If you are using CRF or QP based encoding, yes the filesize will change

But according to that mediainfo report, the he's using 2pass bitrate. The filesize will be the same. The difference is the quality will become lower as you shorten the GOP size, unless that bitrate is high relative to content complexity (unlikely for 12364kb/s for a "typical" blu-ray)

[Surlias]

Unlike VLC and MPC-MC it doesn't look like XBMC has that setting. I guess you're stuck re-encoding with keyint 24.

Too bad. At any rate, I did a full encode at keyint 24 and that seems to have completely fixed the issue with seeking in XBMC.

Out of curiosity, does anyone else encode with a low keyint value like 24, and what has your experience been with quality?

One oddity though is that it doesn't seem to be consistent across all my encodes. Primarily it seems to affect longer movies. Encodes that are shorter than around 2 hours seem generally unaffected, though I haven't done an exhaustive test of all my sub 2 hour encodes. What difference should run time make?

Incidentally, I just noticed the full encode I did is a sub 2-hour movie, so maybe it's not a good example of whether keyint 24 is a real fix or not.

[poisondeathray]

In today's age of huge HDD, who really cares about a few % better compression ?

24 frame length GOP is the standard for full BD. You can officially use 48 with (2sec) with "long GOP" settings if you want slightly better , if you keep the bitrate <15000 --vbv-maxrate 15000 --vbv-bufsize 15000 . I'm guessing it will work for you XBMC setup, since it will work on retail BD setups, but this makes some assumptions about XBMC.

runtime shouldn't affect it at all

--keyint is the most important factor by far when talking about seek issues.

Other things can affect it (but less likely) are bframes (you should keep it 3 or less, similar to BD standard - but you have it set at 6)

The other is bitrate variances, spikes. This can cause issues with playback. If you see to a point that is non buffered it can exhibit issues with buffer underruns. But this usually manifests as stuttering in playback (eg. you seek to a point and it begins to stutter). You can reduce issues by setting the vbv settings (just like you would for a BD compatible encode)

[Surlias]

If you are using CRF or QP based encoding, yes the filesize will change

But according to that mediainfo report, the he's using 2pass bitrate. The filesize will be the same. The difference is the quality will become lower as you shorten the GOP size, unless that bitrate is high relative to content complexity (unlikely for 12364kb/s for a "typical" blu-ray)

Are you saying the bitrate I'm using is too low?

[poisondeathray]

If you are using CRF or QP based encoding, yes the filesize will change

But according to that mediainfo report, the he's using 2pass bitrate. The filesize will be the same. The difference is the quality will become lower as you shorten the GOP size, unless that bitrate is high relative to content complexity (unlikely for 12364kb/s for a "typical" blu-ray)

Are you saying the bitrate I'm using is too low?

No , I'm saying if you use a bitrate based method of rate contol, you will always get the same (or close to) filesize .

The reason is Filesize = Bitrate * Running Time

It was partially in response to jagabo's pretty graphs. Those don't really apply in your case since you used 2pass bitrate

[Surlias]

You can officially use 48 with (2sec) with "long GOP" settings if you want slightly better , if you keep the bitrate <15000 --vbv-maxrate 15000 --vbv-bufsize 15000.

I'm not clear on what you mean by "long GOP" settings. Is that what the indicated vbv-maxrate and bufsize are?

[Surlias]

No , I'm saying if you use a bitrate based method of rate contol, you will always get the same (or close to) filesize .

The reason is Filesize = Bitrate * Running Time

It was partially in response to jagabo's pretty graphs. Those don't really apply in your case since you used 2pass bitrate

Oh I see. However, his graph does help with finding the "sweet spot" that I can reduce keyint to without too much of an impact on quality with my 2pass encoding. Looks like it's around 64. Is that an acceptable keyint? Or is it better to keep it roughly in multiples of the frame rate?

[poisondeathray]

You can officially use 48 with (2sec) with "long GOP" settings if you want slightly better , if you keep the bitrate <15000 --vbv-maxrate 15000 --vbv-bufsize 15000.

I'm not clear on what you mean by "long GOP" settings. Is that what the indicated vbv-maxrate and bufsize are?

In the context of BD, the GOP size for film is 24 . That's standard 1sec rule. So there is an IDR frame at the beginning of each group of 24 (or even more frequent, that's the upper limit). The "long GOP" is a variant that trades off longer GOP interval (for better compression). ie, 2sec or 48 vs. lower bitrate. This tradeoff is you are limited to < 15Mb/s . This is because early BD chipsets have limitations in processing power. Again this is in the context of BD, it may or might not apply to XBMC.

[jagabo]

If you are using CRF or QP based encoding, yes the filesize will change

But according to that mediainfo report, the he's using 2pass bitrate. The filesize will be the same. The difference is the quality will become lower as you shorten the GOP size, unless that bitrate is high relative to content complexity (unlikely for 12364kb/s for a "typical" blu-ray)

Of course. But you can see that using a GOP size of 32 will give you ~12 percent less quality than a GOP size of 256. Or conversely, he'll have to use ~12 percent more bitrate to get the same quality.

[poisondeathray]

If you are using CRF or QP based encoding, yes the filesize will change

But according to that mediainfo report, the he's using 2pass bitrate. The filesize will be the same. The difference is the quality will become lower as you shorten the GOP size, unless that bitrate is high relative to content complexity (unlikely for 12364kb/s for a "typical" blu-ray)

Of course. But you can see that using a GOP size of 32 will give you ~12 percent less quality than a GOP size of 256. Or conversely, he'll have to use ~12 percent more bitrate to get the same quality.

Yes I see it's a relationship you can extrapolate from. But it's important to note that same quantizer or CRF value does not indicate same "quality" when settings are changed. Yes CRF in general is a rough estimate of "quality", but the x264 authors have emphasized that CRF does not indicate "same quality"

[Surlias]

I've got an encode queued using keyint=64 (Hobbit EE) for testing purposes. Before I potentially waste ~20 hours of CPU time doing that, can anyone give their opinion on whether this is a keyint value worth trying?

[poisondeathray]

I've got an encode queued using keyint=64 (Hobbit EE) for testing purposes. Before I potentially waste ~20 hours of CPU time doing that, can anyone give their opinion on whether this is a keyint value worth trying?

Have you tried looking in XBMC specific forums ? They probably have "recommended settings" posted

[jagabo]

Before I potentially waste ~20 hours of CPU time doing that, can anyone give their opinion on whether this is a keyint value worth trying?

Going longer than 64 won't give you much better quality/compression -- you're look at low single digit percents. With bitrate based compression you probably won't even see the difference between 32 and 64 without looking at enlarged still frames. And even then, you won't be able to see which is better without comparing to the source.

The length of the movie doesn't really matter. What matters is how many long shots there are. You'll typically get a keyframe at each shot change. So with lots of short shots there won't be much difference between long and short GOPs.

Oh, you don't seem to know what a GOP is. A GOP is a "group of pictures" -- all the frames from one key frame to the next. Ie, keyint=64 means 64 frame GOPs (in the absence of other factors that may cause a shorter keyframe).

[_Al_]

I've got an encode queued using keyint=64 (Hobbit EE) for testing purposes. Before I potentially waste ~20 hours of CPU time doing that, can anyone give their opinion on whether this is a keyint value worth trying?

It is kind of waste of your time in any case, because you choose 2pass encoding.

[Surlias]

It is kind of waste of your time in any case, because you choose 2pass encoding.

Is this your way of saying that CRF is the only viable encoding method?

[ndjamena]

It's possible, but it's also true that if it's just a test case you might as well use CRF and save yourself some time.

[Surlias]

It's possible, but it's also true that if it's just a test case you might as well use CRF and save yourself some time.

Ahhh good thinking. Definitely going to do that

[hello_hello]

It seems kind of odd to me that XBMC chokes on standard keyint settings, being a software player. I'm not saying that's not the case, just that when playing video using my PC (MPC-HC mostly) it's never been an issue for me. Could it be a decoding problem? Although even then I've decoded with ffdshow/LAV with DXVA/CUVID or CPU decoding and not had a similar problem. I have SD 50p video encoded with a keyint of 500 and my TV's media player isn't bothered.
When you refer to "seeking" you're referring to jumping from point "A" to point "B" and not to problems with fast forwarding or rewinding?

Yes I see it's a relationship you can extrapolate from. But it's important to note that same quantizer or CRF value does not indicate same "quality" when settings are changed. Yes CRF in general is a rough estimate of "quality", but the x264 authors have emphasized that CRF does not indicate "same quality"

Whether there's a direct relationship between the increase in bitrate for a given CRF value and any decrease in quality for a specific bitrate I don't know, but it seems there's likely to be one (I ran some keyint test encodes myself a while back and from memory keyint 24 increased the file size by about 10%).

Although while typing this I thought about how "tuning film" increases the bitrate for a given CRF value compared to "tuning none" as it tends not to blur as much, and therefore when applying the same logic it'd mean for a given bitrate the film tuning must decrease the quality compared to tuning none, and that made my head hurt a bit so I stopped thinking about it.....

[Surlias]

XBMC has keymaps for jumping back and forward in time. The ones I use most are "back 7s", "back 30s", "forward 30s", and of course, chapter skipping. Also, sometimes I use the "jump to a specific time" function. The problem I'm having is that at some point (but not right away) during playback these actions start to fail and instead of skipping back 7s or forward 30s or to a specific time or to the next chapter, it just kind of stutters and then keeps playing, and I have to stop and restart playback to get time skipping to work again. I suppose it very well could be an issue with my install of XBMC, but I haven't been able to dig up any possible solutions. At any rate, the first keyint=24 encode I made never glitches out. But maybe I should be seeking a possible solution with XBMC first and foremost.