Hello all,
I'm looking for opinions on what would be the best CPU out there to build an encoding machine. I'm thinking of getting a dual CPU mother board and to put 2 quad cores on it (if such a MB exists). It must accept windows XP. I encode from Avisynth using heavy filters to lossless as well as FLV, MPEG, WMV, Xvid...
Also, I would have a direct LAN connection with my current computer to lower network induced data transfer rate reduction. Is it a good idea? I thought it would allow me to control both computers easily with same monitor-mouse-keyboard using the "remote connection" in windows.
Overall, I need to set things up to take advantage of the soon to come new processing power... How would you set things up?
Thanks!
+ Reply to Thread
Results 1 to 29 of 29
-
-
What is your price range? You can easily spend $10K on a setup
Tyan released a quad socket board for Shanghai (AMD) processors (16 physical cores). You have many choices for dual socket boards, but Intel's dual socket gainestown platform isn't out until next year. You can buy their current harpertown processors (2P) now.
Most of the video software is not optimized for more than 2-4 cores. The bottleneck is usually avisynth filters (and avisynth MT filters are buggy). In this case clockspeed is the only way to improve performance. x264 unfiltered will scale almost linearly to 16 cores using high settings, but any filtering will usually be the bottleneck
In most cases, you will be better off taking a core i7 and overclocking it. (4 physical + 4 logical cores in 1 socket)
You can network 2 computers and use remote desktop or VNC software to control both off 1 keyboard-mouse. Another option is to buy cheap KVM hardware which is cheap for 2-4, but gets expensive as you scale up or need other features. This would only be beneficial if that was the bottleneck (ie. you are doing some network transfers) otherwise it would be a waste -
A quad core CPU running around 3Ghz, 1066Mhz or faster RAM, two or more hard drives would give you quite an improvement over the computer listed in your profile. I also use Gigabit LAN adapters on my computers to transfer data to my servers for storage. The computer I have in my profile encodes fairly fast.
But MKV high definition (Blu-ray to MKV) still takes about six hours. And that's using 100% CPU on all 4 cores. Xvid or other similar formats encode much faster. The transfer rate over the Gigabit LAN is about 40Mb/s, which is a limitation of my hard drives and controller, not the LAN. -
Originally Posted by poisondeathray
I spent about $3000 on my current system from scratch but I am using SATA instead of SAS like some are using with dual-quad systems. Cost would have gone up quickly had I gone with faster processors. I'm limited to using the 1333 FSB Harpertowns with my board but bus speed isn't important for raw encoding and the Adobe apps seem plenty happy with the current system. If you need suggestions on building a dual-socket system let me know, but be sure it's really worth your $$$ before deciding to go that route.
Only a few media encoders will utilize 8 cores. I haven't tried the new encoder that Adobe bundles with CS4 Master but if it's similar to their previous MainConcept iteration I bet it will be fully multi-threaded. The nice thing about the dual-quad is you can run an encoding job on one chip (4 cores) and its own set of HDDs while editing with Premiere from source residing on another drive with not much impact to system speed. It is extremely multi-tasking.
I also use GbE to connect to my file server, but it goes through my GbE switch. I have dual-GbE on both of my workstations so they can move things back and forth as quickly as my drives can serve it. Directly connecting them, when set up correctly, might have some performance benefit depending on your network infrastructure.
How much video work do you do on an average day? Is this for a production environment or personal projects?FB-DIMM are the real cause of global warming -
@rallynavvie - my bad, you are correct about the OS limitations on sockets
You can tapemod the seaburg chipsets to run at 400FSB instead of 333 if you have DDR2-800 FB-DIMMS and a decent board & cooling. This works on Supermicro boards no problem. In fact, most decent boards will have memory dividers so you can even run cheaper DDR2-667 memory. So if you have E5430's @ 2.66GHz, they will run at 3.2GHz instead because the multi is locked.
Since the OP mentioned heavy filtering, I can't emphasize enough that the bottleneck will be the filtering, not the encoding. In this scenario, even adding even a 100 cores will not help you. Only an increase in clockspeed will help you. If the heavy filtering limits you to 1fps, thats all you will encode at. E.g. try deinterlacing 1080p HD AVC content - slow as molasses! Most resize filters are not MT either, or very limited. Once you combine the filters - even slower than molasses!
You can get around some bottlenecks by offloading this to the graphics card (e.g. if you have a cuda enabled Nvidia card - with DGAVCdecNV - it deinterlaces very fast compared to CPU deinterlacing). Hardware deinterlacing can be very fast and high quality -
Thanks for your answers so far!
I think I haven't put enough details on what this computer will do. This is for work use. I make WMVs, FLVs and MPEGs from VOBs. What I do is use DG Index to make a .dv2 and .wav and load this in Avisynth. At this point the heavy filtering comes in. I deinterlace (using MVBob, maybe it's of interest, it's slow but it's by far the one I had the best results with), filter noise, etc. and encode to 640x480 Lagarith to have a good lossless version for all other format encoding. So the bottleneck happens once. The files are usually about 20minutes long. Using the lossless files in Vegas Video is great, I have set up a 4 disk Raid 0 (debatable, but not here :P).
A lot of the times though, I work with shorter clips that I take from the lossless videos. My workflow is often stuck because when I launch some encoding for a 2 minutes video, my computer becomes much less usable, and I need to wait for it to be done, it's not stuff I can setup to run at night. I want the extra speed to make the short encodings much more swift and get working on the next clip quickly. I understand the bottleneck problem. Using MT in Avisynth, I managed to get my AMD dual core to run at about 80% average. Maybe I can get something out of a quad, or at least of a much faster dual core. So many possibilities!
About the hardware deinterlacing, how good are the results? I think MVBob is astounding, can it get close to it, or better? That would drop one slooow filter from my script and speed up the looooong first pass in avisynth.
Money: That would be 2000$ max (the case with all it needs). I don't think going harcore with the HDD is necessary, neither do I need insane storage and I don't work with HD or full length files. 2 Raptors would be good I think, I have other means of storing the final products.
Other things: KVM would be too much, I don't deal with Blu-Ray to MKV -
Yes MVBob and TempGaussMC_beta1 provide excellent deinterlacing in the avisynth world, but are slow! What's worse is that they don't really scale with parallel processing so only an increase in clockspeed will help to alleviate the bottleneck.
DGAVCdecNV is only for AVC sources right now, but neuron2 is working on support for MPEG2 and possibly VC1 sources. Early tests show that it's better quality than TempGaussMC_beta1 if you can believe that. Have a look at the Doom9 thread for examples. Furthermore it frameserves at 40-60fps (vs 1-2 fps if you were doing it with CPU on HD sources) - so it looks to be very promising to alleviate the deinterlacing bottleneck
Again the bottleneck in your workflow is the filtering. The only way around it is clockspeed, possiblity hardware offloading (in the future with neuron2's work), and selectively choosing faster filters (e.g. mplayer resize in ffdshow is much faster than LanczosResize in avisynth). Hopefully avisynth-MT development will progress as well, because none of the filters are that well multithreaded currently.
Upgrading to a quad core will be a definite improvement in your workflow & multi-tasking over a dual core. -
Would it be wise to look for a dual core with a faster clock speed than a quad core to save some bucks?
-
I think it's better to get a quad in your situation given your tasks.
You can get the older q6600 for ~$200 now and cheaper DDR2 memory (like 2x2GB for ~$75). These overclock very easily with a decent motherboard, even on stock voltages. 99% of them will go from 2.66=>3.2 just by bumping up the bus speed from 333=>400
Personally if I was faced with your situation, I would get a Core i7-920 and overclock it a bit. This new platform is more expensive (enthusiast boards are around $350-$375, and it only uses DDR3 which is more expensive ~$300 for 3x2GB - it's triple channel, not dual channel; but the CPU is about $350). But it is much faster at well multithreaded apps. e.g. ~30-40% faster with x264 encoding and 3dStudioMax. It's a bit more "future proof" -
Originally Posted by Gargalash
In fact, a well multithreaded encoder would simply suck up 100 percent of any additional cores so adding more won't make the system more usable. -
I agree with PDR - I7 920 - It's twice as fast as the Q6600
"Quality is cool, but don't forget... Content is King!" -
Why are the originals in VOB? Can't you get the source footage? Your results are going to be so much better and you will have more control over the resulting products if you can get the source.
FB-DIMM are the real cause of global warming -
Good good! I like what I hear. Thanks for all your ideas.
Any opinions on overclocking the i7920? -
Well retail i7-920 's were just released yesterday (at least officially), and there are dozens of accounts of easy overclocks if you search.
This will be the one I get when prices settle a bit. Retailers tend to jack up the prices on launch, and limited availability doesn't help.
Warning: Definitely do your research and read up on proper overclocking, cooling, voltage etc.. before you try it. It can be safe if you do it properly with mild to moderate overclocks; but this probably isn't the best forum to discuss it in. -
@rallynavvie:
No we can't get the source.
@poisondeathray:
I will sure be carefull if I end up trying overclocking. I already found something on the matter and it looks like it is quite easy to give a significant boost! -
Originally Posted by Gargalash
for years i ran a successful pest control business, which brought me into contact with a wide range of people, including a few in the recording and film business. on of my clients was this guy that lived in nyc, that had a recording studio and small video production studio in his home. we got to talking and as he was giving a tour of his setup i noticed 2 computers, both apples, 1 dual 800 mhz g4 and a dual dual core g5, which at the time was the fastest machine apple made. i also noticed something quite odd, all of his recording and video equipment was connected to the dual g4, so i asked him why that was. he responded by telling me because the dual g4 was faster than his quad core g5, something i found impossible to believe.
he must have seen that look on my face because he quickly added that the dual g4 had a $2500 sound card and a $10000 video editing card, as well as scsi drives and since it would have been a major pain in the ass to transfer all the hardware and software to the g5 (which would have resulted in minimal performance gains), he used the g4 for work and the g5 was a play thing.
the moral of the story? if you are willing to spend the dough to build a high end encoding machine, you are probably better off spending the money on a hardware encoder with included software rather than a dual socket motherboard, smp capable cpu's and registered memory.
perhaps something like this:
http://www.optibase.com/MovieMakerHD/
http://www.sonic.com/products/Professional/SonyMPEG2Encoder/screen.aspx
these are just 2 examples, albiet on the high end of the cost spectrum.
if you shop around you can find a few others that are much more reasonable; trying to build an encoding pc without a hardware encoder is just silly, my phenom 9500, with 4 gigs of ddr 2 667 takes roughly 9 hours to encode a 90 minute movie to 1920x1080 (mpeg-2) at a bitrate of about 10 Mb/s, even if you could build a dual socket pc that was 5 times as fast, you would still be just shy of real time encoding.
add filters, frameserving and h264 to the mix and you are still looking at encode times that are measured with an hour glass.
hardware encoding boards are the only way to go... -
Hardware encoders are on their way out. With most professional editing suites being made to properly multi-thread and how cheap multi-core processors are there is no need to spend thousands on dedicated hardware anymore. 5 years ago I'd recommend a hardware encoder for serious work, but then NLEs still had their place back then. Now if you're processing 16 different streams at once then it would help to have a hardware encoder but for most editing solutions that's overkill.
As for workstation video cards it depends. Now with Adobe's suite taking advantage of regular desktop cards there are only so many apps that run much better on Quadros and such. I had an FX1100 for a while and only really noticed better performance with Illustrator over the 7600GT that eventually replaced it. They still give better performance to CAD and 3D modeling programs though and are still worth their price there.
FWIW I don't recommend overclocking or RAID 0 for a production environment. The data bandwidth from RAID 0 isn't needed unless you're working with several files at once in raw HD formats. Overclocking, though easier and safer than it has been in the days of old, is still a risk. This risk is further exacerbated by the heat you will have to dissipate during long encoding sessions. Expect a shorter lifespan for that system even if you are able to deal with the heat issues. I got 5 years out of my old dual-socket workstation, I expect another 5 years from this one. I think the amount of use I get out of the money spent is well justified.FB-DIMM are the real cause of global warming -
Can the hardware encoders do all the filtering and processing as well? or do they just function strictly as encoding engines? If your filtering limits the frameserving input to low fps, a hardware encoder won't help much in that situation. Furthermore, MPEG2 encodes fast, especially at SD resolutions in the example Gargalash gave
My view is a bit different than rallynavvie's on overclocking. Of course you do so at your own risk, and YMMV, but benefit outweighs the cost IMO. Intel Core2 chips have huge headroom. AMD chips, not so much. I've done this for years, and some of my systems still run 24/7 100% load doing DC projects for 4-5 years now. This is more intense than just encoding or video processing which is a much less intense stress load.
A mild 15-30% overclock can be done with same stock voltage, using a decent aftermarket cooler maybe $30-40 will let you run 15-30 degrees cooler than the stock cooler under load. So that's same voltage, 15-30 degrees cooler....hmmm. Of course if you used that cooler on default speeds it would be even cooler. Not overclockign is like having a Ferrari's and driving in 1st gear all day long. I've also run phase change systems (-20 to -30 degress) under high voltage and high overclocks, but that definitely shortens the lifespan...trust me LOL
Buying the cheapest chip in the series (usually ~350 for the 2.66 GHz chips and overclocking mildly it to 3.2) saves you ~$700 that you could spend on a better graphics card, or save for your next upgrade. In this industry, the pace of innovation means your rig is outdated within 6months. Frequent upgrades are the key, and selling your old stuff before it devalues too much. For example, a single core i7 with mild overclock destroys my old 2 socket Harpertown in most benchmarks. The Harpertown cost 2x as much and is only 1 year old. Sure core i7 is great now, but the 32nm Westmere die shrink is ~1year away...frequent upgrades are the key. -
Humm, hardware would indeed be overkill, plus I have a better idea of what I might get now, and we're talking about 1500$.
Another question: Let's say something from neuron2 becomes available and I can use the GPU to deinterlace, what card would I be happy to have at this point? Nothing overkill is needed, as I'm only working with VOBs.
Thanks for all your ideas! -
The setup I have listed in my computer details works fairly well. You could go with a Q6600 CPU to save a few dollars. I use a ATI 3870 video card and it's relatively inexpensive, but does well with MKV/Blu-ray playback. I overclock the 2.5Ghz CPU to 3.0Ghz. That gives it similar performance to a CPU that costs twice what it did. It has good cooling, so temperature is not a problem. It's finishing a six hour MKV encode at present, at 100% CPU utilization on all four cores, CPU temp is 46C.
With Vista, it scores 5.9 on all settings, so I'm happy with it.The cost was about $700 total.
-
Originally Posted by Gargalash
-
Originally Posted by poisondeathray
Also I have not seen any benchmarks showing a single socket i7 beating a dual-socket quad-core system in anything that is multi-threaded. Marks for single-threaded applications have almost always favored single-socket platforms (gaming for instance) due to the architecture limitations present in workstation/server environments.FB-DIMM are the real cause of global warming -
Originally Posted by rallynavvie
For professionals who rely on CPU power and spend a lot of their time in front of the computer - e.g. maybe video professionals, CAD, 3D modelling - this strategy makes sense. If you cut your h264 encoding time or rendering time by 40% that is huge! Spending a day or half a day doing a project? That's a no-brainer to me... That alone will cover your upgrade costs in no time at all! Time=money
Regarding benchmarks, I said mildly overclocked core i7; and there are plenty of benchmarks that show this. I run most of my quadcores ~3.6 and my Harpers are BSEL modded 2.66=>3.2. Most run 24/7 100% load.
Here is a chess benchmark
Source:
http://www.xtremesystems.org/forums/showthread.php?t=202139
Note entries 17 and 19. A single socket i7 beats a dual socket skulltrail Harper at the same clockspeed. There are many more benchmarks where the case is similar, although the Harpers don't always take that bad of a beating. Again, it's application dependent.
1) TogaII142JD-8cpu 21/47 02:05 132.628.680 8.434.120 +0,25 Nehalem @4.2Ghz(Core i7 965ES) DDR3: 8-8-8-20 Rol-co
2) TogaII142JD-8cpu 21/46 02:59 185.524.894 8.198.883 +0.20 Core i7 940 @4.2Ghz DDR3: 10-9-9-26 Revogamer
3) TogaII142JD-8cpu 21/52 03:39 216.935.295 8.111.273 +0,18 Nehalem @4.0Ghz(Core i7 965ES) DDR3: 7-7-7-18 Rol-co
4) TogaII142JD-8cpu 21/48 02:44 164.649.166 8.016.970 +0,28 Core i7 920 @4.0Ghz DDR3: 7-7-7-21 Chri$ch
5) TogaII142JD-8cpu 21/47 02:21 135.337.470 7.670.355 +0.20 Nehalem @3.85Ghz(Core i7 920ES) DDR3: 8-8-8-19 Monstru
6) TogaII142JD-8cpu 21/45 02:17 130.670.694 7.599.623 +0,21 Nehalem @3.875Ghz(Core i7 965ES)DDR3: 9-8-8-24
7) TogaII142JD-8cpu 21/46 03:46 208.317.723 7.345.841 +0,26 Skulltrail 2xQX9775 @4.2Ghz DDR2: 4-5-5-17
8) TogaII142JD-4cpu 21/47 02:51 135.822.644 6.908.635 +0.20 Q9650 @4.51Ghz DDR3: 6-5-5-12
9) TogaII141SE-16cpu 21/45 03:30 94.511.341 6.843.413 +0.22 Gainestown 2xNehalem @3.07Ghz DDR3: 9-9-9-24
10) TogaII142JD-4cpu 21/44 02:25 117.841.680 6.787.092 +0.20 Q9650 @4.5Ghz DDR3: 6-6-6-15
11) TogaII142JD-16cpu 21/46 03:59 96.988.402 6.521.362 +0.26 Gainestown 2xNehalem @2.93Ghz DDR3: 8-8-8-19
12) TogaII142JD-8cpu 21/54 02:52 134.125.742 6.509.767 +0.19 Skulltrail 2xQX9775 @4.189Ghz DDR2: 4-4-3-9
13) TogaII142JD-4cpu 21/46 02:12 106.263.958 6.413.397 +0.23 Q9650 @4.4Ghz DDR3: 8-7-7-21
14) TogaII142JD-8cpu 21/46 03:08 151.186.585 6.411.283 +0,24 Nehalem @3.2Ghz (Core i7 965ES) DDR3: 9-8-8-24
15) TogaII142JD-8cpu 21/52 03:31 174.872.390 6.286.188 +0.25 Skulltrail 2xQX9775 @ 4.0Ghz
16) TogaII142JD-8cpu 21/49 03:29 174.282.748 6.089.768 +0.20 Skulltrail 2xQX9775 @3.59Ghz DDR2: 4-4-3-9
17) TogaII142JD-8cpu 21/46 03:54 172.582.215 5.891.624 +0.30 Nehalem @ 2.93Ghz
18) TogaII142JD-4cpu 21/48 05:17 116.220.879 5.493.977 +0,32 QX9650 @4.0Ghz DDR3: 7-7-7-18
19) TogaII142JD-8cpu 21/47 03:50 154.531.874 5.419.762 +0.21 Skulltrail Harpertown 2xE5440 @ 3,067Ghz
20) TogaII142JD-4cpu 21/43 02:48 56.606.900 5.299.045 +0,17 QX9650 @3.8Ghz DDR3: 7-7-7-18
http://www.xtremesystems.org/forums/showthread.php?t=206572
A big problem I have with the Harpers is the slow memory. FB-DIMMS are high voltage, run super hot, have high latency and are SLOW. It's the Achilles heel of that platform. All applications that require some decent memory performance are impaired. The Gainestown should fix this. Upgrade time again. -
Originally Posted by rallynavvie
if hardware encoders are on their way out why is it that there are more than a few companies that are getting ready to market cell processor based hardware encoders and why is it that ultra high end video editing/encoding pc's, machines that cost close to 100 grand all come equipped with hardware video encoding/editing boards?
have you ever tried to encode 1080p h264 or vc-1 video at 20+ Mb/s? -
Originally Posted by Gargalash
how about you just spend $290 and have a pc that can encode faster than any cpu based software encoder:
http://www.engadget.com/2008/10/03/leadtek-intros-spursengine-packing-pci-e-card/
faster than real time HD encoding of H264 and mpeg-2, sounds like a winner to me, not to mention a much smarter solution than buying a dual socket xeon setup with new power supply, ram, etc.
oh, and i practice what i preach, i am currently trying to find a retailer where i can buy this card (if anyone has a link to a seller, post it here).
edit: from leadtek's site:
http://www.leadtek.com/eng/tv_tuner/overview.asp?lineid=6&pronameid=447 -
[quote="deadrats"]
Originally Posted by Gargalash
I should have clarified my statement: the prosumer (and consumer) PC market does not need expensive hardware encoders. $300 is a nice price point for a card like you linked as long as it works with all the right apps. However cards that cost thousands are for the few NLE and professional builds out there and at that point you're also looking at blade farms and several computers to leverage the power of those specialized systems rather than just a sole desktop.FB-DIMM are the real cause of global warming -
Originally Posted by rallynavvie
but i'm talking about encoding 1920x1080 at maximum quality settings at 20 Mb/s and up, i really doubt your setup could do even mpeg-2 encoding in real time under these conditions let alone h264. -
Fo Gargalash's specific case, the filtering and processing are the bottlenecks, so even encoding at 1000fps won't help much. He's encoding MPEG2 SD frame, which is fast on modern PC's. But as you said, full 1080p high quality AVC is much different story...
But, if that card can do 1/2 of what it claims to do I'd buy it. I tend not to trust marketing slides, other recent hardware (mis)adventures include AVIVO and Badaboom. Canopus has a similar card called the FIRECODER blu, which is just a rebadged OEM board, but I think the hardware is otherwise similar
Image quality is important. If it's encoding at some AVC Main profile with low quality settings I could care less. This card has 1/2 the #cores each running at 1/2 speed of the Cell so I am highly suspicious of their claims. Furthermore the software implementation is critical here - from what I gather it is very limited. If you can't effectively leverage the full use of the hardware, or if you are limited to certain quality levels - that would be very dissappointing.... So perhaps that can be improved in the future, I don't know.
Here are 2 reviews (Google translated from Japanese)., and it's about 10x faster encoding AVC which is amazing...but no image quality comparison, not details, or perhaps something lost in transalation. If you recall, using equivalent settings, CPU was faster than Badaboom and AVIVO with better quality...
http://translate.google.com/translate?u=http%3A%2F%2Fpc.watch.impress.co.jp%2Fdocs%2F2...-8&sl=ja&tl=en
http://translate.google.com/translate?u=http%3A%2F%2Fpc.nikkeibp.co.jp%2Farticle%2Fnew...-8&sl=ja&tl=en
Regardless, if you be the "guinea pig" deadrats, and pick up one of these I'd love for you to post some testing results. If the claims are even remotely true I'd pick one up too!
Cheers -
I need a computer to encode to FLV, WMV, and MPEG, all for web use, from VOBs. Before that, I use bottlenecking Avisynth filters. How would the hardware encoder help me? I do need a higher clockspeed I think, only for the Avisynth filters, it is justified. Also, I repeat that I don't deal with HD, transport stream or anything like it.
Can a hardware encoding card do very high quality very fast deinterlacing and output it in a lossless AVI format? That would be a good point to consider it. But then I am still far from done at this point: I still need to apply more avisynth filters and I am still looking for faster encoding of my WMV and FLV.
Similar Threads
-
Ideas or tips for building computer (mostly for video editing/encoding)
By robjv1 in forum ComputerReplies: 13Last Post: 14th Jun 2011, 14:21 -
Building a PC for Video Editing/Encoding - Need Advice plz!
By MaTTuP in forum ComputerReplies: 9Last Post: 6th Nov 2010, 10:37 -
Is my computer powerful enough?
By T French in forum Newbie / General discussionsReplies: 8Last Post: 18th Apr 2010, 14:54 -
SUPER converter program has Encoding Failure Error
By kerrex_2006 in forum Video ConversionReplies: 4Last Post: 26th Sep 2009, 13:44 -
Video_TS to Divx. super poor quality after encoding
By mistervolcom88 in forum MacReplies: 5Last Post: 13th Sep 2007, 13:49