External HD Dropped Frames - Enclosure or HD???

[tarrickb]

Hey guys...
I'm capturing DV from my ADVC-100 (DV) to the firewire port on my laptop and then to my WD 120GB (7200 RPM, 8MB cache) HD. The hard drive is connected to the laptop via USB 2.0 in a Vantec NexStar 3 external enclosure.
When I capture, I get spontaneous batches of dropped frames. This means that i don't get one or two dropped frames here or there, they get dropped at 85-225 frames at a time. Sometimes, the capturing program(Sony Vegas and Sceneanalyzer) crashes, and sometimes its able to recover and continues capturing.

I don't think my laptop is the problem, because when i go through my computer, much of the same happens.
So, my question, do you guys think its a problem with the controller in the external enclosure, or my old (but fairly fast) HD?

EDIT: Another point to note is that the HD never has anything on it before i start capturing, and about 40% of the time, i can go 8 hours without any dropped frames.

[whitejremiah]

i'd say you should try a differant capture program....vegas is a GREAT editing program as i understand, but as far as capturing.....im not too sure, i havent heard terribly GREAT things about it, although i havent heard much horrible about it either....i'd suggest taking a stab at maybe virtualdub or winvcr instead of vegas, then importing your captured files into vegas for editing?

ideally though, you should try to avoid capturing to an external hdd if at all possible, as the data has to travel through more cables and such to get to the harddrive.....

[tarrickb]

Well, i also tried Scenanalyzer which is supposed to be good for DV capture.
I wish i could move away from the external HD but it's the only solution when using my laptop because of the unimpressive 60 Gig HD (30 available).

[edDV]

Hey guys...
I'm capturing DV from my ADVC-100 (DV) to the firewire port on my laptop and then to my WD 120GB (7200 RPM, 8MB cache) HD. The hard drive is connected to the laptop via USB 2.0 in a Vantec NexStar 3 external enclosure.
When I capture, I get spontaneous batches of dropped frames. This means that i don't get one or two dropped frames here or there, they get dropped at 85-225 frames at a time. Sometimes, the capturing program(Sony Vegas and Sceneanalyzer) crashes, and sometimes its able to recover and continues capturing.

I don't think my laptop is the problem, because when i go through my computer, much of the same happens.
So, my question, do you guys think its a problem with the controller in the external enclosure, or my old (but fairly fast) HD?

EDIT: Another point to note is that the HD never has anything on it before i start capturing, and about 40% of the time, i can go 8 hours without any dropped frames.

Capture the DV stream to your local drive and then copy the file (secure file copy) to the external drive.

USB2 and OS processes are clobbering your stream and there are no retries with DV transfer. Lost frames are lost frames. It's a stream not a file transfer.

[tarrickb]

Capture the DV stream to your local drive and then copy the file (secure file copy) to the external drive.

USB2 and OS processes are clobbering your stream and there are no retries with DV transfer. Lost frames are lost frames. It's a stream not a file transfer.

Just wondering if there are any solutions as I often record 8 hours at a time, and much prefer using my laptop w/ external HD rather than my pc with all 8 of its fans!

Also wondering if you could further explain what the secure file copy process is all about.
Thanks.

[tarrickb]

Also, like i said, i don't get dropped frames ALL the time....just often. For example, i just finished capturing 3 hours of DV without a single dropped frame, so it happens quite randomly.

So, if i don't experience any dropped frames, are there anyother disadvantages to using and external HD (quality etc...???)

[edDV]

Capture the DV stream to your local drive and then copy the file (secure file copy) to the external drive.

USB2 and OS processes are clobbering your stream and there are no retries with DV transfer. Lost frames are lost frames. It's a stream not a file transfer.

Just wondering if there are any solutions as I often record 8 hours at a time

DV from the ADVC-100 is a constant ~28-34Mb/s stream that needs to flow into the HDD without interuption. If the path is blocked you loose frames because there is no significant hardware buffering. Programs like WinDv add some software buffering but the path is long to an external HDD and many things can interupt the flow.

The path to the internal drive is shorter (something like this - simplified)
ADVC->IEEE-1394 card->PCI bus->Memory->PCI bus->ATA Disk Controller->HDD

The internal HDD is being shared with the OS, background tasks, other PCI devices, etc. All this needs to stand back so the data can flow to the file on the drive. If any process blocks the bus or takes priority to the drive, DV frames are lost.

The path to the external drive is longer and potentially more risky. The USB2 disk controller relies on a software process running on the CPU.
ADVC->IEEE-1394 card->PCI bus->Memory->PCI bus->USB2 interface(PC)->USB2 interface (enclosure)->ATA Disk Controller->HDD

Fixed USB2 interface.

...and much prefer using my laptop w/ external HD rather than my pc with all 8 of its fans!

DV transfer should take as close to zero CPU load as possible so no heat.

Also wondering if you could further explain what the secure file copy process is all about.
Thanks.

Once the DV stream is stored in a DV-AVI file, then the Windows OS will mange file transfers as packet data and any transfer errors result in the OS resending packets. Thus the "copy" is secure.

Editing and encoding can be done from files on the external drive with security. The worst you will see is jitter in playback but the data is secure.

[edDV]

Also, like i said, i don't get dropped frames ALL the time....just often. For example, i just finished capturing 3 hours of DV without a single dropped frame, so it happens quite randomly.

So, if i don't experience any dropped frames, are there anyother disadvantages to using and external HD (quality etc...???)

It all depends on whether you can live with occasional drops. If you have zero drop tollerance, you will take more extreme measures.

Dropped frames can be managed if the process priorities can be controlled. I used to use a different hardware profile and user identity for DV file transfers on my old PIII notebook. Everything was stripped from that profile except what was needed to transfer.

For example: networking, audio, ports (other than IEEE-1394 and USB in your case)...

[tarrickb]

I'm definatly going to try different hardware profiles.
Hopefully it will help. Thanks.

[edDV]

I'm definatly going to try different hardware profiles.
Hopefully it will help. Thanks.

On the software side, all backgrond tasks including antivirus. At the extreme shut down most XP services.

Start->Run-> msconfig

[vhelp]

terrickb, I would try other DV capturing software. As many as you can
try. For me, it all depends on what I'm doing. But, I do use winDV for
most of my capturing needs. But i do use others as well.

But, I'm confused with your setup.

I too use an external encloser for my HDD, and it was specially for the
use of capturing (transfering) through DV to the HDD.

My setup is like this:
** AMD XP 1800+ CPU
** ESC KS75A motherboard
** USB-2 External HDD (by ADS)
** Canopus ADVC-100
** WIN98

and I do NOT drop any frames in my captures, even when going out to my
USB-2 enclosure kit. ( but if you are, then it's probably on account mainly
by your audio cards' driver support for DX and DS [directshow/directX] or
that you have a lower verion installation issue )

-vhelp 3855

[edDV]

I think something like an antivirus scan or temperature calibration on the drive, or some other service is taking over the PCI bus thinking the system is idle. Can easily happen over an 8 hour capture.

[tarrickb]

well i'm going to try all the suggestions tonight but unfourtunatly i may have encountered an even BIGGER problem.

I just noticed today that one of my 8 hour captures (from the USB HDD) has jittery video. There were NO dropped frames reported during the capture. The individual videos were carved out of that 8 hour capture, and resaved as DV on another internal drive on my main computer. This is the first time i have noticed the video jittering. It is a slight jitter that occurs about every 5 seconds of video.
Any ideas what this could be from???????????

P.S. All of my drives have been defragmented, and the jitter occurs in all of my media players.

[vhelp]

you know.. you never did say what your source is.

** vhs
** cable/satellite tv
** other

Also, I don't see any reason why you should capture 8 hours straight in the
first place. I would never do this, no matter how good my system specs
might be. (this leads me to believe that your source is vhs, but I could
be wrong)

If you have 8hrs of source to capture, do it in 1hr increments. You may
be pushing your setup beyond the limits it was intended for.

-vhelp 3856

[tarrickb]

My source is satellite TV.
I capture in large chunks because I do projects involving music videos, so i record a channel that plays nothing but that.
It hasn't been a problem in the past, just with this batch it seems.

Looking at the video again, i would re-describe it more as jerky video, and it seems to occur every 2-3 seconds (not 5 as mentioned above).

[tarrickb]

Here is an example:

http://s44.yousendit.com/d.aspx?id=0TTCW5BPIDPVY09SA8A8I4GY45

please check it out and let me know what you all think.
Anyway possible to fix this??? its important because seeing as this stuff was captured live, i can't get it back (unless its aired again while i'm capturing).

[tarrickb]

Anything guys???
i would really appreciate some advice on this problem.

[edDV]

Explain what you see is the problem.

Typical PVR quality. Are you saying this hurts your eyes?

[jagabo]

I saw lots of dropped frames (previous frame duplicated, sometimes 3 or 4 times in a row).

[vhelp]

I noticed (after the d/l 'ed clip) that there were a lot of
dropped frames. Obviously you have a frame-drop problem
here. Then I got ta thinking..

Question..
Are you capturing (via transfering) your DV (satellite) source
to external HDD, but adding in the mix, a real-time MPEG capture,
in an effort to cut down on the process of over-night encoding ??

I ask, because it was odd that you went through the process of posting
an MPEG clip, vs. something smaller, like divX or something.

EDITED post
-vhelp 3860

[tarrickb]

well the program didn't report ANY dropped frames.
Do you think it could have been dropping frames and not detecting it?

[edDV]

How do we know that the dropped frames weren't intended?
Try a clip we can duplicate from other sources like a popular commercial* or cable/TV show.

[vhelp]

I revised my previous post, but in case it was missed..

(i could be wrong, but)

Sounds to me like you got a real-time mpeg going on there. That would
be the reason you are dropping frames in this case, and given your
laptop setup. You might be pushing your setup beyond its limits when
you throw in a real-time mpeg.

-vhelp 3862

[edDV]

vhelp has got a good point

Your 8hr capture file is 8x13.5= 108GB right?

The fraction of that indicates lost frames.

[jagabo]

I believe the ADVC-100's behaviour is to substitute duplicate frames when frame's are dropped (due to sync loss or whatever) to keep the video in sync with the audio. If this was happening the computer wouldn't know frames were being dropped.

[tarrickb]

it's weird because, i know dropped frames are dropped frames, and it's not too much of a problem as long as the system is able to recover and continue recording (the 1 or 2 seconds of missing videos usually occurs during a commercial ).
As i said, this is the first time this has happend and from what it seems, it's not happening anymore.
I'm just a little worried because, it renders my videos unusable and i really don't know what the cause was and hope it doens't happen again.

Some of the videos that were affected were quite important and cannot be re-captured, so i know its not likely, but is there anyway to improve these jerkey videos???

Thanks.

[tarrickb]

Oh and to answer the posts above.
NO i am not doing a real time encode to MPEG 2.
I just chose an MPEG-2 video stream because i was lazy and didn't want to venture outside of vegas (also knew the file size wouldn't be that big becasue clip was short).

Guess i should have chosen .wmv....ah well!