Use a software codec when capturing with an ADVC300?

[timothyd_1975]

Any help much appreciated.
I am capturing 20 S-VHS video tapes via an ADVC300 unit. Although I am editing on PPro CS2 I am using a software called SCLive to create the .avi files generated by the ADVC300.
I note from GrassValley's website that the ADVC does actual employ its own proprietary DV codec. While pc-side my sofware SCLive offers about five codecs, among them DV Type 1 and DV Type 2, and does not offer the option of capturing uncompressed.
My question is this: - if the ADVC is already performing .avi compression with its own codec, should I be capturing uncompressed when the signal gets to the pc? Ie dumping this SCLive program for something like VDub and capturing in uncompressed mode? I would assume that two different compression altorighms being applied at encoding time would cause degradation?
Until recently I naively assumed that because .avi is a container file that compression is never applied to it. If anyone knows of any articles that would help fill in the blanks in my knowledge, any links would be muchly appreciated.
Once again, thanks for any help.
Tim

[edDV]

Any help much appreciated.
I am capturing 20 S-VHS video tapes via an ADVC300 unit. Although I am editing on PPro CS2 I am using a software called SCLive to create the .avi files generated by the ADVC300.
I note from GrassValley's website that the ADVC does actual employ its own proprietary DV codec. While pc-side my sofware SCLive offers about five codecs, among them DV Type 1 and DV Type 2, and does not offer the option of capturing uncompressed.
My question is this: - if the ADVC is already performing .avi compression with its own codec, should I be capturing uncompressed when the signal gets to the pc? Ie dumping this SCLive program for something like VDub and capturing in uncompressed mode? I would assume that two different compression altorighms being applied at encoding time would cause degradation?
Until recently I naively assumed that because .avi is a container file that compression is never applied to it. If anyone knows of any articles that would help fill in the blanks in my knowledge, any links would be muchly appreciated.
Once again, thanks for any help.
Tim

Whoa partner.

Is ScLive ScenalyzerLive ?

Are you trying to do optical scene detection?

The Canopus (GrassValley) ADVC-300 has an internal hardware codec that will dump straight to your hard disk in DV format.

Talk to me. Processing on import is mostly destructive.

[Cornucopia]

My 1st answer would be "DV Type-2", but you might want to check to see if it's transcoding to a software codec with that option (you DON'T want that).

Unless Scenalizer/SCLive has a particular feature that you can't live without, I'd suggest you use something more direct (DVApp, DVIO, WinDV, Virtualdub, or...Why don't you just cap in PProCS2, since you say you've got it! Much more direct, and then it's already loaded in the media database)

Scott

[timothyd_1975]

Thanks for the feedback edDV / Cornucopia

Yes I am using ScenalyzerLive. I picked ScenalyzerLive because it is really easy to use to cut and combine .avi files.

I am effectively re-importing all my source files. Before I got the ADVC I attempted to use a TV card to perform the capture. This created MPEG2 files which PPro didn’t like very much.

Unfortunately I only descovered that PPro doesn’t like MPEG2 files after I’d done all the clip logging. So now I have to re-create the source files and attach them to an existing project. (Hence SCLive – easy to ‘slice and dice’ those .avi files).

So if the ADVC delivers straight to the hard disc, surely I still need to employ some software to write the datastream to the disc, irrespective of whether it employs compression?

I will try using PPro to do so. Im not at home now so I can't check that Ppro will not employ some codec of its own.

So would you agree with me in saying I don’t need to encode on the pc as the ADVC is already doing the job?

Cheers guys

[lordsmurf]

Huh?

You use Scenalyzer to grab a DV stream from the Canopus ...
... and then you somehow have an MPEG-2 that you're trying to import?

What'd I miss?

[edDV]

Let's try to explain.

S-VHS is Y/C and S-Video delivers Y/C to the Canopus ADVC-300.

The Canopus ADVC-300 converts analog Y and C to digital YCbCr. While doing it, the ADVC-300 can do minor line TBC, noise reduction and gain corrections (from buttons on the panel).

The output from the ADVC-300 is a DV stream over IEEE-1394 that can be captured as is to a DV-AVI file with software as simple as WinDV. Just start capture and stop capture.

Simple software detects scene breaks from discontinuities in time code or from manual assist. Scenalyzer's claim to fame is to attempt optical scene change detection from picture analysis. The processing is only to find the scene break, not to alter the stream. This is useful for continuous timecode captures from videotape or tuners rather than camcorders.

Does that make sense to you? There should be no processing going on to the digitized capture other than the ADVC-300's settings.

[timothyd_1975]

Yes it does. Thanks edDV.

So long as when I use Ppro or WinDV or SCLive, if I select the DV option I am not envoking some kind of unwanted codec?

Reason I ask is I have read it is possible to capture from a device such as a mini DV camera straight to hard disc via FiWi to a file of type .dv. I would imagine if one hour of DV-AVI is about 13.5Gb and this is supposed to be 5x compression that the resultant .dv file for one hour would be 5 x 13.5 = 67.5Gb per hour.

Im not suggesting I take S-VHS which is only about 420 lines and store it at 67gb per hour. I guess Im just wondering if I do simply need to trust in the ADVCs DV encoding, then select DV on whatever software I'm using? (And not try and get a software that captures raw).

Sorry - I may be being a bit thick here.

[edDV]

Yes it does. Thanks edDV.

So long as when I use Ppro or WinDV or SCLive, if I select the DV option I am not envoking some kind of unwanted codec?

They all start and stop the DV stream and capture the data to a DV-AVI file. The only codec use is for monitoring video and audio. The DV stream itself is not affected.

Reason I ask is I have read it is possible to capture from a device such as a mini DV camera straight to hard disc via FiWi to a file of type .dv. I would imagine if one hour of DV-AVI is about 13.5Gb and this is supposed to be 5x compression that the resultant .dv file for one hour would be 5 x 13.5 = 67.5Gb per hour.

No. .dv (called raw) is the DV stream without the AVI wrapper. The data in the stream are the same.

Im not suggesting I take S-VHS which is only about 420 lines and store it at 67gb per hour. I guess Im just wondering if I do simply need to trust in the ADVCs DV encoding, then select DV on whatever software I'm using? (And not try and get a software that captures raw).

DV format is the same regardless of the codec used (hardware or software) or the file wrapper. Apple uses .dv, Windows uses DV-AVI. The only capture choices involve Type1 vs. Type2 and audio sample rate and bit depth. Type 1 vs 2 is also audio related. Type 2 offers audio as both interleaved and as a separate stream. This is more efficient for editing but increases the size of the resulting file.

[timothyd_1975]

Got it! Thanks very much for that edDV.
I am happy with my workflow now. Ive messed this up enough in the past. I didn't want to make any more costly mistakes. This project is well overdue for completion.

[Cornucopia]

Also, as a point of understanding:

DV gets as its source (pre-DCT compression) 4:1:1 Y-Cb-Cr(or alternately, 4:2:0 for PAL?) Std Def. The DCT compression is 5:1. It's no longer RGB.

If you do the math...

R+G+B @ 8bits/pixel/color is 24bits/pixel. 4:4:4
Y-Cb-Cr is like unto that, but each pixel may or may not be sampled, depending upon the color subsampling choice (Y--luminance/greyscale is ALWAYS sampled for every pixel). So 4:4:4 is really a sampling RATIO. Think of it this way: for every 4 pixels, Y is sampled 4 times (1 for each pixel), with 8bpp. Similar ratio works for Cb+Cr (or Cb/Cr Vert. + Cb/Cr Horiz.)

Thus 4:1:1 is sampled for Y once every pixel (8bpp), for Cb/Cr once every 4 columns and once every 4 rows. That works out to:
8 + (8/4) + (8/4) = 8 + 2 + 2 = 12. That's 1/2 the bitrate of 24bit RGB, without REALLY losing anything noticeably visible to the naked eye, and BEFORE DCT compression.

So if 720x480 * 29.97fps * 24bpp (or 3Bytes/pixel) = 237Mbps, 4:1:1 with DCT 5:1 divides that by 10 = 23.7Mbps. For a cushion, round it up to 25Mbps. Add in audio, control data, metadata, some error correction and you've got 36Mbps.

That's your number with which to calculate transfer rate by.
For STORAGE, the 25Mbps is probably closer to your final size. Figure 10-13GB/Hour.

That's whether it's raw DV (meaning, without being in a container and without container headers), or in an AVI container, or in a Quicktime container. Neither adds THAT MUCH overhead compared to the hugeness of the data file. RAW here does NOT mean uncomressed.

Scott

[timothyd_1975]

So Scott when you say "DV gets as its source (pre-DCT compression) 4:1:1 Y-Cb-Cr(or alternately, 4:2:0 for PAL?) Std Def. The DCT compression is 5:1. It's no longer RGB." - correct me if im wrong, but if DV’s source is 4:1:1 Y-Cb-Cr then this is already sub-sampled and it is not RGB? If so i presume RGB is used when the image is recorded to DV tape?

At what point is the image actually stored as RGB? Somewhere I presume if we can conclude that final compression is one tenth the data of the original image.

I found an article via wiki (Douglas A. Kerr) that explained J:a:b notation so I feel I am on top of subsampling now. Though I would like to understand more about the allocation of bits to pixels in RGB encoding - ie what's the difference between 16/24/32 bit RGB?

I am also considering the best video color depth to use in VDub for capturing from the ADVC, which is supposed to deliver DV II - which would be 4:2:0. If I pick '4:2:0 planar YCbCr (YV12)' - (where i presume YV12 refers to 12 bit storage) - then I would hope that the Application correctly identifies where this subsampling has already occurred on the ADVC and doesn't further subsample the picture?

I searched on DCT and became quite confused and confronted with a throwback to trigonometry in high school.

I suspect I may be over thinking this one. If there are any articles you feel I may benefit from reading, don't hesitate to just throw me a hyperlink in lieu of further explanation. Either way, thanks very much for your time!

[edDV]

So Scott when you say "DV gets as its source (pre-DCT compression) 4:1:1 Y-Cb-Cr(or alternately, 4:2:0 for PAL?) Std Def. The DCT compression is 5:1. It's no longer RGB." - correct me if im wrong, but if DV’s source is 4:1:1 Y-Cb-Cr then this is already sub-sampled and it is not RGB? If so i presume RGB is used when the image is recorded to DV tape?

At what point is the image actually stored as RGB? Somewhere I presume if we can conclude that final compression is one tenth the data of the original image.

Typical camera flow.

1. Camera sensor(s) optically capture RGB.
1A. Some broadcast cameras output pre matrix RGB for chroma key.

2. Camera matrix section converts RGB to YUV or YCbCr (if digital)
2A. Some cameras output pre-encoded YCbCr as SDI or HDMI*

3. Camera Image Processor

4. Resample chroma to 4:1:1 (NTSC) or 4:2:0 (PAL or HDV)

5. Encode to DV or HDV

6. Record to tape and/or output IEEE-1394.

7. Play tape with output to analog and either DV or HDV over IEEE-1394.


* Depending on camera design this live output can be from step 2, 3, or 5. Tape playback is from step 7 (encoded).


The ADVC-300 does this

1. Input composite or S-Video (Y/C). Composite is Y/C separated with comb filter.

2. Chroma is decoded to Cb and Cr

3. TBC/Proc/Noise Reduction (switchable)

4. Resample chroma to 4:1:1 (NTSC) or 4:2:0 (PAL)

5. Encode to DV (5x DCT compression)

6. Output IEEE-1394.

[lordsmurf]

To suggest the Canopus ADVC 300 has a line TBC is giving it too much credit. It's about as "TBC" as the frame synchronizers in a low-end DVD recorder.

[Cornucopia]

I'll try to do my best to answer these (one at a time)...

#1. I didn't make myself clear enough. Here's a sort of flowchart of the image capture and storage of a DV-cam:

A. Real image (Infinite tone gradation, Wide Dyn. Range, RGB) -->
B. Imager image buffer (8/10/12/14bit/pixel/colorPrimary grad depends on cam choice/often log, Usu. Med/Narrow DR, Bayered RGB) --> DeBayering-when needed, LUT processing, poss. subsampling
C. Imager electronics memory (usu 8bit/color lin grad, same DR, Y-Cb-Cr 4:4:4 or 4:2:2) --> Subsampling, Prep for DCT
D. DCT Input buffer (8bit/color equiv, same DR, Y-Cb-Cr 4:1:1) --> DV-DCT compression
E. DV storage input buffer (same DR, Y-Cb-Cr 4:1:1 12bit/pixel-all colors) --> Control Data/Metadata addition, Multiplexing, Packetizing
F. DV file storage--HD, P2, Flash, etc. (same specs as above), or --> Stream buffering,
G. DV/DVCam/DVCPro/Digital8 tape recording

At NO point beyond #B is there any RGB anymore. ONLY specially modded cams like the DVX-100's "Andromeda" mod allows for bypassing/tapping the output from #B prior to the processing and saving the RAW data via external HD (usually USB 2.0/Firewire/etc). Otherwise the only types of cams that go beyond this are Hi-end Pro Cams (RED or better) that save "RAW" data, usually via HD-SDI to external store.

#2. Standard is 8bits/pixel/colorPrimary. R=8bits, G=8bits, B=8bits. Total =24bits. 8bits give 256 levels of gradation. If this were 10bit/pixel/color, or 30bitRGB, each color would have 1024 levels of gradation. Therefore, much less banding. This doesn't, however, improve the dynamic range. That is fixed by the type of imager + possibly optics used.
When RGB gets processed, edited, and effects added, it is VERY IMPORTANT to not worsen the banding. This is quite like the old analog generation loss. So often RGB is converted from 8bit/pel/color to 10, 12 (36total bits) or 14 (42bits) or 16 bit (48bits).

Do not confuse this with HDR (High Dynamic Range) imaging which REQUIRES higher bitdepths, which vary in many formats (16bit Linear, 32bit linear, 16 and 20bit log., 32bit float).

Either type needs many more bits, so that when the image is finally transferred back to a standard distribution image type, it WON'T have lost anywhere near what would have happened had it hadn't been upgraded 1st.

#3. The ADVC-300. The Firewire output of that is exactly like the output of a DVtape at stage #G above. Y-Cb-Cr 4:1:1 (4:2:0 is ONLY for PAL versions). That's 12bits of storage/pixel (aka YV12). But raw YV12 isn't compressed 5:1
Don't save as YV12, because you'll be saving the decompressed DV data in 5x the space of the DV stream or file. And with the Decompression, any editing will invariably require either saving to a equal or better format to avoid generation quality loss (and hence incurring a MAJOR size penalty) or going BACK to DV compression (and incurring that said generation quality loss).
Save the capture directly to DV-AVI. Any self-respecting editor will have enough smarts to leave the DV packets alone and pass them through bit-for-bit identical and avoiding that Decompress+Recompress quality loss (unless of course you are adding procesing/effects,in which case you should expect a minor amount of Qual Loss IN ANY SYSTEM. And it won't be as noticeable, because the subject has been "modified".

You kind of ARE overthinking this. Don't worry. Do this...

Analog cam or VCR --> best connection available (Component, then S-Video, then Composite) --> ADVC --> Firewire --> PC adapter card firewire port --> DV capture app (WinDV, DVIO, DVapp, or editor's DV/Capture module--Vegas, PPro, FCP, AVID, Edius, etc). --> save captured DV stream into DV-AVI. Type 1 or Type2 depends more upon which apps you'll be using for the next stages of editing, etc. and what they prefer. Edit, resave as DV.

When you've got a finished DV master program, then will finally come the time to decide how you're going to distribute/view it as a consumer. MPEG2 for DVD, MPEG2/VC1/h264 for BluRay (don't even need to upconvert, just keep in SD rez and let the machine do it on output), Vp6/vp7/vp8/Sorenson/h264 for Flash, Divx/Xvid for DivX players, MP4 for iPhones/3Gs. And on and on.

HTH,
Scott

>>>>>>>>>>
edit: Damn, edDV you beat me to it while I was editing all that crap.

[Cornucopia]

((OT: pertaining/supplemental to above))

To help with the bits/DR difficulty, think of this analogy: Flights of stairs in a building relate to the various levels of light sensitivity/brightness values.

A standard 8bit/pixel/color setup (which BTW is predominantly LowDR) aka 24bit RGB would be like 2 flights of steps where each step is 1 foot higher than the next. And the bottom floor is 1st floor, top floor is 2nd floor.

12bits/pixel/color RGB (aka 36bit) would be like the same 2 floors, but this time the steps are each 6 inches higher than the previous. More steps but same overall distance. Closer to a "sliding board/ramp" than the previous setup.

HighDR would be like a building where the steps are 3 inches different in height and there is a basement and a 3rd and maybe 4th floor. LOTS MORE variability.

Scott

[edDV]

>>>>>>>>>>
edit: Damn, edDV you beat me to it while I was editing all that crap.

No problem, we came at it from different perspectives.

Since we are doing this in depth I'll add a few more sidebar topics.

* CCIR-601 (ITU Rec-601) derived formats (e.g. D1, DV, MPeg2 DVD, etc.) define Y as:

8 bit 256 total levels (e.g. D1, DV, DVD, HDV, AVCHD, ATSC, DVB, XDCAM, HDCAM)
with black defined at level 16, nominal white at 235 with 236-255 reserved for overshoot.

10 bit 1024 total levels (e.g. D1, Digital Betacam, HDCAM-SR, AVC-Intra, Red)
with black defined at level 64, nominal white at 940 with 941-1023 reserved for overshoot.

As Cornucopia said the added levels of 10 bit or more are for computational accuracy (e.g. for filtering, compositing or effects) through a multi generation process, not for "dynamic range".


* NTSC DV/DVCPro and PAL DVCPro use 4:1:1 chroma sampling for better multi generation performance in broadcast workflows. 4:2:0 was intended as a distribution rather than production format but recent edit software has reduced 4:2:0 chroma generation loss for PAL DV, HDV and XDCAM.


* Production formats generally use 4:2:2 chroma sampling (e.g. D1, DV50, Digital Betacam, DVCProHD, HDCAM-SR).
D1 format also allows for 8 or 10 bit YCbCrA 4:4:4:4 when using two recorders. HDCAM-SR format allows 4:4:4 RGB in 8 or 10 bits (linear or Log). 4:4:4 sampling allows for better color accuracy (e.g. Pantone color match) and better filter or translational effects performance.

[timothyd_1975]

edDV & Cornucopia thank you very much for those explanatory notes. its taken me a few days to soak that up. i now feel i am in command of my encoding process (and quite amazed at the quality of picture you can get from DV considering all the compression it goes through).

i have further reading to do re: DCT. for now i am happy to conclude that it is a mathematical formula that results in lossy compression (shame).

my concern around encoding results from getting poorer results with what should be a top notch setup. as mentioned before i am using the ADVC300 & (now) PPro to capture with input from Sony SVO 5800P high end S-VHS deck (second hand). my first capture setup was composed of a consumer JVC S-VHS deck and Compro USA VideoMate TV card DVB-T300 with the card's sofware creating MPEG2 files at the highest bit rate possible. when i compare the avis to the mpegs, the mpegs look nicer (not as 'harsh', more detail, better colour). (Nb both deck connections were made with s-video connection).

i suspect the Sony deck i am using is no good, despite head cleaning. i found the manual on the web. i learned you can enter a code to find out how many hours the unit had edited for, and learned that it was over the recommended hours. shame considering it cost me $500AUD.

Sony say they can replace the head unit for $3500. So I found another cheaper deck on ebay - Panasonic AG6730P - to try as source. Coming through the mail as I write this.

When I apply the ADVC's noise filter it does clean up the image substantially. However detail and colour are still not as good.

The deck is the last variable to eliminate. Gosh what a journey!

Once again, thanks for your help guys. Most generous.

[edDV]

I'm busy with a project at the moment. I need to read this thread again. It may be this weekend.

[2Bdecided]

The ADVC series give you the luma and chroma exactly as it is received. They are very accurate.

Other devices may change the levels, boost the chroma etc.

My camcorder tapes have incorrectly high luma levels. The ADVC accurately reflects this. Some DVD recorders correct for it. (btw, too much luma also gives the impression of too little chroma!)

Cheers,
David.

[Cornucopia]

Another option for you would be to get a Real Good TBC and then a REAL GOOD Analog Cap card. One that allows you to choose the codec. Choose Uncompressed (or losslessly compressed) 4:2:2. Then you're getting even less compression (and better color spectrum/bitdepth) than DV.
So your internal subsystem has to be able to support the much higher throughput, but if it can, your quality would improve another grade
Scott

[edDV]

edDV & Cornucopia thank you very much for those explanatory notes. its taken me a few days to soak that up. i now feel i am in command of my encoding process (and quite amazed at the quality of picture you can get from DV considering all the compression it goes through).

i have further reading to do re: DCT. for now i am happy to conclude that it is a mathematical formula that results in lossy compression (shame).

DCT is the broad name for intraframe compression similar to that used for JPeg and MPeg. Listen S-VHS is plenty lossy already.

DV is low loss compared to other alternatives and can capture much wider analog bandwidth than is present in S-VHS. This is actually a problem. Out of band noise is also digitized. Ideally one would bandpass filter before A/D like the high end broadcast cards do. That is expensive. Consumer cards over sample and then low pass filter digitally which can very lossy especially when done real time.

my concern around encoding results from getting poorer results with what should be a top notch setup. as mentioned before i am using the ADVC300 & (now) PPro to capture with input from Sony SVO 5800P high end S-VHS deck (second hand). my first capture setup was composed of a consumer JVC S-VHS deck and Compro USA VideoMate TV card DVB-T300 with the card's sofware creating MPEG2 files at the highest bit rate possible. when i compare the avis to the mpegs, the mpegs look nicer (not as 'harsh', more detail, better colour). (Nb both deck connections were made with s-video connection).

Test the system with high quality video in place of the S-VHS deck (e.g. a cable box or digital camcorder). There are possible issues with the player alignment or the S-VHS tape signal quality. Also a TBC is usually used to align the pixels before A/D. This can't be fixed digitally.

Beyond that you just need to compare capture components under controlled testing. Swap one box at a time.

i suspect the Sony deck i am using is no good, despite head cleaning. i found the manual on the web. i learned you can enter a code to find out how many hours the unit had edited for, and learned that it was over the recommended hours. shame considering it cost me $500AUD.

Sony say they can replace the head unit for $3500. So I found another cheaper deck on ebay - Panasonic AG6730P - to try as source. Coming through the mail as I write this.

These decks are expensive to repair.

When I apply the ADVC's noise filter it does clean up the image substantially. However detail and colour are still not as good.

The deck is the last variable to eliminate. Gosh what a journey!

Once again, thanks for your help guys. Most generous.

Missing in your chain is a full featured TBC designed for a VHS deck.

[edDV]

Another option for you would be to get a Real Good TBC and then a REAL GOOD Analog Cap card. One that allows you to choose the codec. Choose Uncompressed (or losslessly compressed) 4:2:2. Then you're getting even less compression (and better color spectrum/bitdepth) than DV.
So your internal subsystem has to be able to support the much higher throughput, but if it can, your quality would improve another grade
Scott

The TBC is critical for getting pixels into vertical columns. Compression algorithms evaluate pixel neighborhoods for motion and object boundaries. When pixels are in horizontal jitter, compression errors result.

[timothyd_1975]

Cornucopia if i were to go the good capture card & good TBC route are there any brands that spring to mind? I did spend several hours looking at some of the thousands of cards listed on videohelp. To be honest there was so much to soak up, if someone could say "Pinnacle is good; so is X, Y and Z" then that would be very helpful.

So my second SVHS deck turned out to be a dud. It butchered my head cleaning tape.

I have noticed quite a few Panasonic NV-MS4 and NV-MS5 movie cameras selling on ebay.com.au at the moment. An NV-MS4 was the camera used to lay down the images back in 1996 - 1998. Question - would using the same type of unit for playback make sense? Obviously whatever nuances an MS4 has handling tape would be the same for other units. And although I suspect they are designed with the recording function being of primary concern, they were deemed to be prosumer level at the time of release. Ie surely playback would be decent?

Cheers