VirtualDub is too fast...need an I/O throttling solution

[markofca]

I'm using VirtualDub to do very basic cut/join edits on DV AVI files. My only problem is that neither VirtualDub nor Windows give me a way to throttle/prioritize disk usage, so VirtualDub immediately consumes 100% of my disk bandwidth in Direct Stream Copy mode.

On the server where I'm doing my edits, I'm also serving other video files to an office with many employees, so every time we're doing an edit on the server, everyone has to stop working (because their video playback becomes choppy as they have to fight with VirtualDub for a pittance of I/O bandwidth). Don't get me wrong, I love how fast VirtualDub is, BUT, the rest of my disk related activity grinds to a near halt.

I'm currently running on a 200MB/sec 4-drive RAID-0 (so nothing slow about it). Getting a faster RAID won't help because VirtualDub will simply consume all of its bandwidth (the only improvement would be that people would have to stop work for a shorter period of time).

So, what I'm looking for is an easy way to throttle VirtualDub's hard drive access. Not many people want VirtualDub to run slower, so I didn't get much help when I threw out a feature request to the author.

I've considered inserting a network between the input and output drives and using the network speed as a throttle, but that's a pretty lousy approach. Vista has I/O prioritization, but it looks like it has to be done by the developer (can't just set an I/O priority on a running task?). I could copy the file out to another non-server machine where the edit is done, but copying a 25-30GB file off to another machine just to edit it then copy it back seems like a huge waste of time. Full processing mode slows things down but only by consuming the CPU instead.

Another option would be to use a different tool to edit the files that had some type of I/O throttling...but I haven't found one.

Any ideas would be appreciated? I'm very close to downloading VirtualDub source code and trying to figure out how to modify it to add a "maximum MB/sec to/from disk" option. That seems the nuclear last resort.

Thanks...

[redwudz]

If you are running XP, open VD, go to 'Task Manager>Applications>VirtualDub, Right click on it and select 'Go To Process', then right click again and set 'Priority' to 'Low'. That may help. It may not slow down it's I/O bandwidth, though.

Someone else may have a better solution.

And welcome to our forums.

[wzrlpy]

Hi,

Try to put your output files on a different physical drive than your input ones (I believe RAID-0 behaves just like a large drive).
Your output drive could be a USB attached hard drive (probably you already have one).

[bigstusexy]

When you go to finally do your work on doesn't virtual dub pop up a box telling you wahts going on with a box at the bottom for process priority?

[jagabo]

Changing the process priority will not make much difference in I/O bandwidth unless there are other CPU intensive applications running.

[bigstusexy]

Most work done on the system will be affected by CPU avaliability this might also affect the IO usage but less so in the case, still worth a shot. Most lower cpu but higher IO uses are usualy direct file moving and copying.


In at least XP we can measure IO reads and write... over a period of time not per second via task manager. Just how many are being taken up/ If you are so close to your IO limit anyway I'd suggest doing it on another system.

[markofca]

Thanks for all the responses so quickly, but changing CPU priority does not help me (either via Windows or in VirtualDub).

CPU is not the bottleneck (in fact, CPU usage is less than 5%). If it were, then process priority would save me. Neither Windows nor VirtualDub have a way to control I/O priority or throttle I/O.

Basically, unless the developer does something special, every process in Windows uses as much I/O as it wants whenever it wants it, regardless of how many other interactive, higher priority tasks are waiting (hence Vista's half-solution of giving developers functionality to prioritize their I/O activity...doesn't help unless the developer has reworked his software to use it).

Using two disks doesn't really help either unless I choose a slower disk to do my edits to and then copy them back to my server (that is, use the slow disk's speed as a throttle). This isn't ideal because it requires an extra copy and slows things down even when it's not necessary (ie when the server I/O is otherwise idle).

Any other ideas out there? Thanks in advance. This is driving me nuts.

[markofca]

One other thing, just to be clear. The reason we're hitting our I/O limit is entirely because of VirtualDub editing. When VirtualDub is not running, the machine isn't close to using 200MB/sec of disk bandwidth (maybe < 10% of that).

A "direct stream copy" in VirtualDub is almost identical to a file copy and goes as fast as the disk can go. Or, in other words, VirtualDub immediately consumes 100% of the I/O bandwidth for itself...no matter how fast the disk is.

[jagabo]

Change to 10 Mb/sec ethernet? A USB ethernet adaptor as a secondary connection to the server?

[bigstusexy]

My only suggestion then is to not do this on the server. What methods have you use to come to the conclusion that its IO?

[jagabo]

QoS scheduling?

http://www.microsoft.com/technet/technetmag/issues/2007/02/CableGuy/?related=/technet/...07/02/CableGuy

[Soopafresh]

Cfosspeed (30 day eval)

[markofca]

jagaboo and soopafresh,

Thanks, these are definitely routes that I'll try. From the Microsoft article, it looks like Vista has good support for this (though complicated, see below). I think I'll give cfos a try first. If I do my VirtualDub processing on a remote machine, but leave the video files on the server, I think I can use cfos to throttle my hard drive usage without having to copy files to and from the server.

Very clever. Hopefully I can get it to work. You guys rock. Thanks again.
---

Policy-based QoS settings in Windows Server "Longhorn" and Windows Vista allow you to prioritize or manage the sending rate for outgoing traffic based on the following conditions:

Sending application (executable path and name)
Source or destination IPv4 or IPv6 addresses or address prefixes
Protocol (TCP, UDP, or both)
Source or destination ports or port ranges (TCP or UDP)

QoS policies are applied to a user login session or a computer as part of a Group Policy object (GPO) that is linked to an Active Directory container such as a domain, site, or organizational unit (OU), or they may be applied to a security group. As part of Group Policy, QoS policies build on your existing Active Directory management infrastructure.

[markofca]

CFOSSpeed didn't pan out. Even if you add Virtualdub.exe as a program in the Misc tab, CFosSpeed doesn't appear to notice VirtualDub's traffic. I believe the reason is that because we're simply writing to a remote disk, virtualdub isn't directly using TCP/IP.

This is being masked by Windows. It appears that CFosSpeed doesn't really provide any support for non-TCP/IP network traffic (ie Windows file sharing), unless I'm not understanding how it works...

[Soopafresh]

Another free one.

http://bandwidthcontroller.com/trafficShaperXp.html


What about kicking the workstations down to 100 megabit rather than Gigabit ?

[markofca]

Thanks for putting me on to this idea. I think I have my solution. Cheap, great interface, and a nifty program for monitoring and tweaking your network.

http://www.netlimiter.com/

It's nearly perfect. Really like the interface. Very easy to use. I can set up limits for incoming and outgoing speeds for any computer on my local network. I've tried it with VirtualDub, and

I can easily throttle VirtualDub's performance at whatever bandwidth I want provided I put my source and destinations across a network, which is pretty simple to do (and probably a good idea).

I think this is a done deal. Thanks again.

[Soopafresh]

Nice find. I'll bookmark that one for a future need.