Long Encoding Times - MPEG2 --> XviD?

[onesikgypo]

Hi,

I have an i7 core PC with 6gb of RAM - and im trying to do a 2-pass encode a 23min show with ivtc(decomb, telecide) to xvid but its taking 57min. When looking at my CPU Usage, its always around 20-30% - i set the threads on my first/second settings to 4 - but that didnt really help.

What could be causing such slow encoding?

[jagabo]

Are you using a multithreaded build of Xvid? The multithreaded builds aren't especially well multithreaded but you should be going faster than that.

[onesikgypo]

i am using: XviD codec v1.2.2

[jagabo]

That version looks like it's multithreaded -- did you set the option (via the "Other Options" button at the bottom of the main Xvid config dialog)?

What frame size and frame rate are you encoding? Right now I'm encoding a 25 fps 720x576 DVD source with Xvid on an Intel Q6600 (no other filtering). The 50+ minute episode looks like it's going to take about 23.5 minutes. Decomb and Telecide aren't that slow I don't think that's the difference.

[onesikgypo]

i set the threads on my first/second settings to 4 - but that didnt really help.

What could be causing such slow encoding?

What frame size and frame rate are you encoding

23.97fps... at 640x352

[jagabo]

I saw your mention of setting threads in the first post but I wasn't sure you were talking about the Xvid setting. What program are you using?

You're doing all your filtering in AviSynth? If you're using VirtualDub or VirtualDubMod do you have it in Video -> Fast Recompress mode?

By the way, my encode was single pass so you'd have to double the time for a comparison. I see about 70 percent CPU usage. Xvid doesn't scale much at all beyond 4 threads. Actually, it doesn't even scale very much beyond 2 threads.

[onesikgypo]

Im using gordian knot, and yes all the filtering is done via visynth.

If you're using VirtualDub or VirtualDubMod do you have it in Video -> Fast Recompress mode?

Does this affect quality?

--

I shoul dnote that the bitrate its encoding at is about 1400-1600

[jagabo]

If you're using VirtualDub or VirtualDubMod do you have it in Video -> Fast Recompress mode?

Does this affect quality?

Yes. And speed. Fast Recompress is faster and better quality than Full Processing.

I shoul dnote that the bitrate its encoding at is about 1400-1600

Bitrate doesn't effect encoding speed much.

Are you doing any filtering besides decomb and telecide? I'll run a quick test with those filters as soon as my current encode finishes (this is the second one I've done since my first post).

----later...

OK, the following script:

MPEG2Source("yadayada.d2v")
Telecide()
TDecimate()
LanczosResize(640,352)

and a 23 minute, 720x480, hard telecined DVD MPG, VirtualDubMod in fast recompress mode, with Xvid set to the highest motion search precision and highest VHQ mode settings, 1 b-frame, Adaptive Quantization, single pass target quantizer 3, 4 threads, took about 12 minutes.

I'm using "Xvid-1.2-127-VAQ.exe" which I downloaded over a year ago.

[onesikgypo]

VirtualDubMod in fast recompress mode

From what i just read, full processing mode is needed to adequately filter/ivtc.

Also, i should also note thats my source is a full 1080i

[manono]

From what i just read, full processing mode is needed to adequately filter/ivtc.

Not if doing all your filtering in AviSynth.

Also, i should also note thats my source is a full 1080i

You should have mentioned that earlier. Then, of course that explains the slow encoding speed.

If downconverting from Hi-Def to Standard-Def, you should also change the colorimetry. The ColorMatrix filter is one way:

ColorMatrix(mode="Rec.709->Rec.601",Clamp=0)

That'll go after the IVTC so you don't have to add Interlaced=True.

[onesikgypo]

I ofcourse understand that a 1080i source takes longer than a dvd source which is typically 720x576 but 57mins encoding time for a 23minute clip, an average bitrate of 1000kbps with an i7 cpu seems a bit long?

[manono]

A 1920x1080 source has 5 times the pixel count as compared to a 720x576 source. You do the math.

You can save a considerable amount of time by resizing the width before doing the IVTC:

MPEG2Source("yadayada.d2v")
LanczosResize(640,Height)
Telecide()
TDecimate()
LanczosResize(Width,352)
ColorMatrix(mode="Rec.709->Rec.601",Clamp=0)

The ColorMatrix filter is fairly slow, but you'll make up that time by using Fast Recompress in VDub.

[onesikgypo]

Yes, but if i do a similar encode with an AMD dual core 2.4ghz - i am getting about the same encoding time, i would exepect in using an intel i7 quad the encoding time be significantly less, or am i making a wrong assumption?

[manono]

You like adding more and more vital information as the thread goes along, don't you, rather than mentioning it all at the beginning? Maybe jagabo wants to take over again. I'm outta here.

[disturbed1]

Xvid does not multithread well at all. Nothing more to it. To get the best out of xvid on a multicore PC, you'll have to run parallel encodes (encode $N of movies = $N of cores at the same time).

If you have an 8 core CPU at 2.0ghz per core, and tell Xvid to use all 8 cores, you might shave 5 minutes off of an hour versus using 1 core. That's just the way xvid works. Divx and ffmpeg (lavc) are both better multithreaded mpeg4-asp encoders. x264 (Mpeg4 - part 10 AVC) has great multithreading. Multithreading in general does not guarantee an increase in efficiency. $N x threads will never = $N x efficiency (12 cores will not be 12 times as fast as 1).

An AMD dual core 2.4ghz being about the same/faster speed as an i7 sounds incorrect. http://www.tomshardware.com/charts/2009-desktop-cpu-charts/XviD-1.2.1,1383.html

[onesikgypo]

Thanks, thats what i was wanting to understand.

Re your suggestion to run multiple encodes at the same time, so that the cores can be more better utilized, are there any tools that will do that, since there seems to be no way for GordianKnot to do this.

Also, re setting video mode to fast recompress - whilst this is doable when doing an encode manually, is there anyway to do this through GordianKnot?

[jagabo]

If you have an 8 core CPU at 2.0ghz per core, and tell Xvid to use all 8 cores, you might shave 5 minutes off of an hour versus using 1 core.

That's not accurate in the absence of heavy filtering. Going from one thread to two in Xvid will reduce encoding time by about 35 percent. Going from one thread to four will reduce encoding time by about 52 percent. The exact difference depends on the particular video being encoded and the Xvid settings used.

https://forum.videohelp.com/topic357460.html#1893993

But since the OP is performing an IVTC on HD material that is where a large part of his conversion time is being spent. Using a multithreaded version fo AviSynth and SetMtMode() should help -- if all his filters are thread safe.