DV Tape vs. SDHC for a HD camera

[edDV]

Poisondeathray,

We are clearly talking past each other here. I'm talking about maximizing original work picture quality using workflow design to distribution. You seem to be talking about recodes at low quality bit rates that one would encounter from web captures or second generation Blu-Ray rips.

For workflow design one starts with the highest quality original source they deem affordable for the project. If I had the budget I'd be shooting HDCAM-SR or AVC-Intra. At the level we are talking here the choices separate around $5000up for DVCPro-HD or XDCAM-EX or $500-$4000 for HDV or AVCHD.

Ideally one would shoot uncompressed or with as little compression as possible. DVCPro-HD and AVC-Intra record individual frames with 4:2:2 sampling at 100Mb/s. XDCAM-HD records in 4:2:0 at 25-50Mb/s using intra-frame and inter-frame compression. AVCHD and HDV record 4:2:0 at 8-24 or 25Mb/s respectively. HDV and AVCHD record 1440x1080i same as broadcast HDCAM format although HDCAM uses 144Mb/s. At 1440x1080i HDV uses 25Mb/s and AVCHD typically uses 8-16Mb/s. Recent models of AVCHD have added a 1920x1080i mode @24Mb/s.

Subjective tests on 1080p monitors show 1440x1080i HDV 1st generation picture quality superior to 1440x1080i AVCHD using similar camera and lens designs. This reflects more the current state or art of consumer level camcorder hardware codecs rather than the potential of AVCHD as a format. But when you buy a camcorder, you can't upgrade the hardware codec. It is frozen in time. Note that at 1440x1080i HDV (MPeg2) is using more than 50% greater bit rate per pixel vs. potentially more efficient AVCHD (h.264).

The most recent higher end AVCHD camcorders allow recording at 1920x1080i @24Mb/s. Camcorder Info reviewed these camcorders as near equivalent to HDV for picture quality at first generation. Note that 1440x1080i HDV is still recording at 50% greater bit rate per pixel. Broadcast engineers learned long ago that bit rate is more important to picture quality than horizontal resolution other things being equal. Consumer marketers knew that the average Joe will think 1920x1080 must be better than 1440x1080. Truth is 1920x1080 AVCHD needs either more bit rate or a better hardware codec design to be superior to HDV.

After choosing your format most picture quality results from lighting and shooting technique. Since AVCHD and HDV use GOP based inter-frame compression, camera stability is a major concern for compression quality as is exposure.

Assuming good field shooting technique, the next major issue for picture quality is minimizing recodes in post production and distribution. And as stated above AVCHD needs more CPU power for timeline search/scrub and processing vs HDV due to higher compression. AVCHD export bit rates are constrained by the Blu-Ray player or PS3. HDV can be can be export encoded up to 35Mb/s for minimal recode loss to Blu-Ray if quality is the goal. If a digital intermediate codec is used, the same could be done for AVCHD source.

[poisondeathray]

edDV - Thanks for the workflow tips, and I agree with everything you've said, except your statement: "I'm saying that on a simple decode-encode h.264 suffers greater recode loss than MPeg2", which is simply untrue.

[edDV]

...
h.264 offers better compression than MPEG2, has clearly better objective measures (psnr, ssim) and subjective scores by a longshot (not even close here). If you were to make a plot, varying the bitrate on the x-axis, with psnr or ssim on the y-axis, MPEG2 never reaches the same quality at any bitrate, and never intersects the h.264 line at any point (h.264 eventually becomes lossless, MPEG2 does not). i.e. at any given bitrate, h.264 will have better quality, or obtains a certain quality level at a lower bitrate. Therefore, subsequent encodes using MPEG2 at the same bitrate will result in greater loss and noise / compression artifacts than h.264.

First note that HDV records with 50% greater bit rate per pixel vs. AVCHD. Second, we are talking about hardware MPeg2 (mature*) vs. h.264 (immature) camcorder codecs. Third, export at source bit rate is only important for AVCHD due to bit rate limitations put on Blu-Ray players and the PS3. HDV source can be exported at up to 40Mb/s MPeg2 depending on the player spec.

[edDV]

Hypothetical situation: Let's say you have a 1080p24 lossless source e.g. Lagarith encode of some CGI animation (or any content)

Scenario A: Encode that source @15Mbps, and then use that as input for subsesquent stage using MPEG2@15Mbps, and one more time (3 generations)

Scenario B: Same thing @15Mbps for h.264 (3 generations)

Which do you think will look better? both objectively (with psnr, ssim scores, difference masks & digital subtraction), or subjectively?

But that is an unrealistic situation in this context. HDV at 25Mb/s seldom needs more than one recode with proper workflow design. 1440x1080 @15Mb/s is unreasonable for MPeg2.

edDV - Thanks for the workflow tips, and I agree with everything you've said, except your statement: "I'm saying that on a simple decode-encode h.264 suffers greater recode loss than MPeg2", which is simply untrue.

This needs to be evaluated with real world hardware and software codecs but I agree with you in theory.

So far all broadcast formats have opted for high bit rate MPeg2 over MPeg4 when recode is anticipated. MPeg4 is only used as the final export codec or for webcast.

[poisondeathray]

First note that HDV records with 50% greater bit rate per pixel vs. AVCHD. Second, we are talking about hardware MPeg2 (mature*) vs. h.264 (immature) camcorder codecs. Third, export at source bit rate is only important for AVCHD due to bit rate limitations put on Blu-Ray players and the PC3. HDV source can be exported at up to 40Mb/s MPeg2 depending on the player spec.

Noted. My statements were always prefaced by "assuming similar input quality." I'm not making a comparison of camcorder image quality, but as you've and many others have said, the top end AVCHD camcorders are comparable to HDV models in terms of image quality. My only concern was your statement about h.264 suffering greater recode loss.

The limitation on bitrate blu-ray media for h.264 is higher than that for DVD media. You can safely use 30Mbps average bitrate with a 40Mbps buffer using h.264 if you use BD25. I think bitrate might be important to discuss for some HDV scenarios, because you can't realistically put HD material on a DVD5 if you use MPEG2, unless your composition is very short. Similarly, your running time is sustantially reduced if you chose to export your HD camcorder material to MPEG2 instead of h.264. The recode loss will more negatively impact encoding to MPEG2 in this scenario, and it's not a completely contrived scenario - many people make home movies.

[edDV]

First note that HDV records with 50% greater bit rate per pixel vs. AVCHD. Second, we are talking about hardware MPeg2 (mature*) vs. h.264 (immature) camcorder codecs. Third, export at source bit rate is only important for AVCHD due to bit rate limitations put on Blu-Ray players and the PC3. HDV source can be exported at up to 40Mb/s MPeg2 depending on the player spec.

Noted. My statements were always prefaced by "assuming similar input quality." I'm not making a comparison of camcorder image quality, but as you've and many others have said, the top end AVCHD camcorders are comparable to HDV models in terms of image quality. My only concern was your statement about h.264 suffering greater recode loss.

The limitation on bitrate blu-ray media for h.264 is higher than that for DVD media. You can safely use 30Mbps average bitrate with a 40Mbps buffer using h.264 if you use BD25. I think bitrate might be important to discuss for some HDV scenarios, because you can't realistically put HD material on a DVD5 if you use MPEG2, unless your composition is very short. Similarly, your running time is sustantially reduced if you chose to export your HD camcorder material to MPEG2 instead of h.264. The recode loss will more negatively impact encoding to MPEG2 in this scenario, and it's not a completely contrived scenario - many people make home movies.

The future will see improvement in AVCHD camcorder encoding plus larger cheaper flash media. Meanwhile I see flash ram XDCAM-EX @25-50 Mb/s coming down to the $1000-2500 prosumer price level. JVC has committed to XDCAM to replace HDV. Canon hasn't announced yet whether it will go with XDCAM-EX or AVC-Intra for future prosumer models.

[poisondeathray]

But that is an unrealistic situation in this context. HDV at 25Mb/s seldom needs more than one recode with proper workflow design. 1440x1080 @15Mb/s is unreasonable for MPeg2.

Fair enough. It's an unrealistic and contrived scenario, but it illustrates the deficiencies in MPEG2. Even after 1 generation, there is substantial loss

What are common HDTV broadcast bitrate ranges? I remember you had an excellent post but I couldn't dig it up...

e.g. don't many Euro channels like BBC use h.264 transport streams avg. ~10-17Mbps , what about in North America?

This needs to be evaluated with real world hardware and software codecs but I agree with you in theory.

This has been, in publications by Universities, and even unofficially on websites. I've even replicated some tests for myself, being the skeptic that I am, I have to see it to believe it. Unfortunately most use PSNR, SSIM but they are the best "objective" measure we have for a subjective trait like "quality"

So far all broadcast formats have opted for high bit rate MPeg2 over MPeg4 when recode is anticipated. MPeg4 is only used as the final export codec or for webcast.

Is it possible that legacy systems /equipment and "edit-friendlyness" play a larger role in this decison?

[edDV]

But that is an unrealistic situation in this context. HDV at 25Mb/s seldom needs more than one recode with proper workflow design. 1440x1080 @15Mb/s is unreasonable for MPeg2.

Fair enough. It's an unrealistic and contrived scenario, but it illustrates the deficiencies in MPEG2. Even after 1 generation, there is substantial loss

What are common HDTV broadcast bitrate ranges? I remember you had an excellent post but I couldn't dig it up...

e.g. don't many Euro channels like BBC use h.264 transport streams avg. ~10-17Mbps , what about in North America?

USA ATSC (like Canada) allows multiple subchannels within the 19Mb/s total channel bit rate. Only the primary needs to be MPeg2 but current TV tuners are limited to MPeg2 in any of 18 resolution/frame rates.

Typical bit rates are:
1920x1080i/29.97 ~16-19Mb/s
1280x720p ~12-14Mb/s
704x480i ~3-4Mb/s

Cable and sat compress more. Cable stuffs 2 or 3 HD sub channels per each ~36Mb/s 256-QAM channel but often uses bandwidth sharing (aka statistical multiplex) to lower average bit rate.

Yes in Europe and other DVB areas, h.264 is an optional local broadcast format. Note that in the USA only the primary channel needs to be MPeg2. The rest of the bandwidth can be used for any format, even data-cast.

So far all broadcast formats have opted for high bit rate MPeg2 over MPeg4 when recode is anticipated. MPeg4 is only used as the final export codec or for webcast.

Is it possible that legacy systems /equipment and "edit-friendlyness" play a larger role in this decison?

Yes, local recode is assumed. Tapes are distributed on 144Mb/s HDCAM or 100Mb/s DVCProHD, etc.
Network feeds are ~45Mb/s*. Local stations add graphics overlays etc, then recode to ATSC transmission bit rate.

* can include multiple program streams.

[edDV]

The limitation on bitrate blu-ray media for h.264 is higher than that for DVD media. You can safely use 30Mbps average bitrate with a 40Mbps buffer using h.264 if you use BD25. I think bitrate might be important to discuss for some HDV scenarios, because you can't realistically put HD material on a DVD5 if you use MPEG2, unless your composition is very short. Similarly, your running time is sustantially reduced if you chose to export your HD camcorder material to MPEG2 instead of h.264. The recode loss will more negatively impact encoding to MPEG2 in this scenario, and it's not a completely contrived scenario - many people make home movies.

Good point. I agree use of MPeg2 for HD to DVDR-5/9 is possible but impractical. Lacking Blu-Ray alternative format support, AVCHD @16Mb/s is the format of choice for direct playback on DVDR media. But these restrictions are absent for BD media or for media players like the Western Digital. One can acquire HDV, edit on digital intermediate and export 16Mb/s AVCHD for DVDR distribution. Then one can encode a higher quality MPeg2 or h.264 or VC-1 for BD release or encode xvidhd or flash for other distribution paths. Seems like the best edit master would be the Cineform Intermediate.

The problem for many "consumers" is they are saving the finished product at too low a bit rate (e.g. 16Mb/s AVCHD or xvidhd) same way they "mastered" SD to low bitrate divx/xvid. They later discover to their dismay that what looked adequate on older displays becomes unwatchable on their newest HDTV.

[poisondeathray]

Thx for the info edDV,

So for things like newscasts , the station graphics / overlays etc are encoded done in real time? or are they pre-produced before with a slight delay before broadcasts? Obviously these are hardware based solutions for both MPEG2 and AVC in Europe?

Just to clarify, all media player based on the same Sigma Designs chip (e.g. WD, Popcorn Hour, Tvix), and BD Players using BD media do have bitrate limitations, and they require a specific buffer size. For BD Players this is based on the linear rotation and angular velocity of the optical media which limits read speed. This is why it is lower for DVD media, which is less "dense". Unbuffered bitrate peaks from buffer underflows cause studdering , and a persistently high bitrate can cause black screen, or no playback at all. Although the theoretical limitation for current AVC playback devices is 62.5Mbps, (as specificed by the AVC High@Level 4.1 Profile), in practice, this is much lower. LOL I made a few expensive coasters early on when I started experimenting with blu-ray. In terms of devices, PS3 is the most robust by far. It can play more higher bitrate non-compliant streams than many of the "flaky" blu-ray players.

For my own projects, I usually use a lossless intermediate because I do a lot of pre-processing with avisynth beforehand. I usually use UT video codec because the encode, and (more importantly) decode function is well multithreaded and much faster then the commonly used lagarith or huffyuv in MT mode, and works well for editing.

(apologies to Cazz for the thread hijack! )

[edDV]

Thx for the info edDV,

So for things like newscasts , the station graphics / overlays etc are encoded done in real time? or are they pre-produced before with a slight delay before broadcasts? Obviously these are hardware based solutions for both MPEG2 and AVC in Europe?

Local broadcast studios usually are live uncompressed with incomng uncompressed graphics and compressed video elements (various formats). All is hardware composited in a device called a production switcher. Welcome to my world.

http://www.engadgethd.com/2007/08/30/what-it-takes-to-produce-an-hd-newscast/

The 24x7 on air integration center is called "master control". It is here that local live studios, network feeds, commercial breaks and recorded video server elements (various formats) are live composited in a hardware master control switcher. The uncompressed output is encoded to ATSC MPeg2 and fed to the transmitter. This done for the primary and each subchannel of the ATSC transmitter feed.

[Cazz]

(apologies to Cazz for the thread hijack! )

no no that is ok.
This is good information to know but I have to tell I dont understand so much about what you two have write.

so if someone can write a summary of all this I going to be very happy

Just one question (Maybe is a very stupid question but I like to be sure)

What kind of tape are I going to use, I mean, can I use same tape that I recording with a DV camera as I recording with a HDV camera??

[edDV]

What kind of tape are I going to use, I mean, can I use same tape that I recording with a DV camera as I recording with a HDV camera??

MiniDV tape comes in various grades and all will "work" in an HDV camcorder.

Standard consumer MiniDV tape (like you buy in a grocery) ~$2-3ea
http://www.tapestockonline.com/son60minprem.html

Pro grade MiniDV tape (as used by professionals) ~$3.00ea (~$2.50ea on sale)
http://www.protapesonline.com/product.php?productid=232&cat=13&page=1
These have tougher construction intended for heavy duty use in fast wind pro tape decks. They also have fewer dropouts and better magnetic signal to noise. These days they are just a little more expensive than standard grade if you buy online.
This is my standard tape for DV and HDV.

"HDV" MiniDV tape. ~$9-12ea
http://www.tapestockonline.com/sohdvmidv63m.html
This is premium HDV labeled consumer tape. These have somewhat better magnetic signal to noise and low dropout vs. the tape above but due to retail markups, sell for about 3x more. Use these for your daughter's wedding or other "important" projects.

Premium "Master" tape ~$16ea.
http://www.tapestockonline.com/sodvdima40mi.html
Use these for your Sundance Festival project.

It should be noted that since DV and HDV are digital formats, the "picture quality" of all tapes above is the same. The differences are primarily the robustness of tape backings, magnetic dropout performance* and the signal strength** of the RF recording.

* All DV and HDV camcorders/players have electronic dropout correction. Since HDV is GOP based (one full frame for every 15) it is possible for a dropout to cause loss of a portion of the I frame affecting a full 1/2 second GOP. In these rare cases the electronic correction will be more obvious.

** This is important for playback on a different transport than the one that recorded the tape.

[Cazz]

ok well I going to buy my in sweden (I think that going to cost to much to freight going to be to much )

So the tape dont have any class level (ex class 1, 2, 3..)so I have to see after name and that.

[edDV]

Pro tapes are sold by pro dealers, not in stores. Best to find an online source.

Try to find Panasonic AY-DVM63PQ from an EU online retailer. Stock up when they go on sale. Pros buy these in bulk.

You should find Sony’s DVM-63HD HDV tape in higher end retail stores but expect to pay more than online price.

[Cazz]

ok, I was looking at

Panasonic DV 60 Super Linear Plus
and
Panasonic DV 60 Linear Plus

that is a little more cheap

But I going to look after Panasonic AY-DVM63PQ


/Update

Did find some Panasonic AY-DVM63PQ for 4,90 euro in finland

[Cazz]

edDV:
Just one little question.

For me is MPEG2 DVD or SVCD and not HD
I have see that you wrote MPEG2 a little diffrent (MPeg2) so I maybe think wrong about MPEG2 or that is maybe something else that call MPeg2

[edDV]

edDV:
Just one little question.

For me is MPEG2 DVD or SVCD and not HD
I have see that you wrote MPEG2 a little diffrent (MPeg2) so I maybe think wrong about MPEG2 or that is maybe something else that call MPeg2

I don't understand the question.

We have been discussing AVCHD vs. HDV for high definition consumer camcorders (under US$1000).

DV is a sequence of 720x576i frames with DCT compression within each frame.

DVD is 720x576i/p MPeg2 (or MPEG2 same thing).
https://www.videohelp.com/dvd

HDV is 1440x1080i MPeg2

AVCHD is 1440x1080i MPeg4 (h.264) or 1920x1080i MPeg4 (h.264)

[Cazz]

sorry it was a little offtopic about this thread.

I did never think MPEG2 can be in HD (HDV)
I know MPEG4 can be HD and some other format but never MPEG2

I going to buy a Canon HV30 but I was just curious about that MPEG2 can even be in HD (Like I did write before I was only thinking MPEG2 can be DVD or SVCD)

[edDV]

sorry it was a little offtopic about this thread.

I did never think MPEG2 can be in HD (HDV)
I know MPEG4 can be HD and some other format but never MPEG2

I going to buy a Canon HV30 but I was just curious about that MPEG2 can even be in HD (Like I did write before I was only thinking MPEG2 can be DVD or SVCD)

Yes MPeg2 is the basis of most HD television including HDV, XDCAM (Sony's Broadcast format used by networks), ATSC HD broadcasting (Americas), DVB broadcasting (PAL countries) and Blu-Ray.


Sony XDCAM MPeg2 HD broadcast camcorder

Advanced Video Codecs (AVC h.264 and VC-1) are newer more compressed formats but not as well developed. They are secondary choices to MPeg2 now and will gain share in the future as they continue to be developed. H.264 is an alternative choice to MPeg2 for DVB broadcasting. VC-1 and h.264 are alternative codecs to MPeg2 for Blu-Ray. VC-1 is targeted more for highest quality applications where MPeg2 is currently being used. H.264 is scalable to many lower bit rate applications for consumer video down to low bit rate streaming applications (phones, digital cameras, webcasting).

For camcorder use, we've been discussing the merits of MPeg2 vs. h.264 for editing content. Generally MPeg2 is much easier to edit on a typical home computer. h.264 is more difficult to edit but because it is more compressed, it can be stored more easily to flash RAM.

Canon HV30 HDV (MPeg2)


Canon HF S100 AVCHD (h.264)