best quality playback and capture for preserving PAL Hi8 camcorder footage?

[HuBanfiT]

Greetings VideoHelp (first post here),

I have about 80-100 90-minute Hi8 prosumer-quality cassettes recorded with Canon UC-X50Hi/X55Hi camcorders over the years (1999-2006) with PCM stereo audio.

The subject matter is contact sports: detailed pictures together with intense motion are sought; footage was often shot under less than ideal lighting.

I would now like to digitize them in the best quality possible to bring them into the modern digital video era for archival and for dissemination.

(See sample of current technical quality here - I'm looking to achieve better: https://www.youtube.com/watch?v=KIQQPyr-_Pc )

My tentative plan:

• investing in or hiring playback equipment
• investing in or hiring or building capture equipment
• capturing the analog PAL Y/C signal at full resolution, preferrably at 10 bits or more (raw baseband Y and C if necessary/possible)
• into a full resolution, lossless data format
• with precise ADC levels/ranges setup (to ensure proper shade and colour rendition)
• coping with tape issues (small amount of dropouts, small amount of dirt, some wrinkled tapes, sync issues when fighters bump into/kick the tripod or from starting/stopping recording - each round of each fight is recorded separated by a start/stop)
• if feasible, multiple playbacks averaged (e.g. play and capture the same video multiple times to average out playback noise)
• followed by advanced high bitdepth processing in Vapoursynth:
• timebase correction (either in hardware, or willing to develop in software - whichever is better)
• PAL decoding in software (if needed)
• best quality motion-adaptive deinterlacing (willing to develop in software)
• best quality motion-adaptive noise reduction (willing to develop in software)
• any necessary conversions (upscaling, encoding) for wide dissemination in the current era
• e.g. best quality motion-adaptive framerate upconversion (from PAL 50Hz to computer monitor/youtube/internet 60 or 59.94 Hz - willing to develop in software)
• any other suggestions/advice you might have

What are your suggestions? What would be my best options?

Thank you in advance.

[johnmeyer]

Easiest option is to have them transferred at Costco, Walmart, etc. They sub out to YesVideo.

However, for 80-100 tapes, that might get expensive.

You can get a Canopus ADVC300 Advanced Digital Video Converter and use that to transfer the tapes. All you do is hook one end of this to your computer, and hook the other end to the analog (audio/video) outputs from your 8mm or Hi8 camcorder. You will need a Firewire/1394 capability in your laptop or computer. If your camcorder is still working, you can use that and not have to buy anything except the ADVC300 and, possibly, a Firewire Express card (laptop) or PCI card (desktop).

Then, once you have the files on your computer, use a non-linear editor (NLE) to edit out the bad stuff, group things together, etc.

I would concentrate first on capturing the video and getting it onto a hard drive. The gadget I recommended creates DV video files which take 13 GB/hour. Therefore, 100 tapes = 1.3 TB, so a garden-variety 2 TB drive (<$100) will hold all of your video. Once you have the video captured, you can go to phase two which is editing and, optionally, trying to slightly improve the quality. Don't expect quality improvement miracles. I've done hundreds -- perhaps thousands -- of hours of VHS, Beta, and 8mm captures, and have developed software for reducing noise, eliminating dropouts, and removing chroma aberrations. Those things you can improve. What you cannot do is get detail where there wasn't any, or make the audio suddenly sound like studio-quality audio.

[SameSelf]

When it comes to Hi8 video assuming your camcorder has S-Video out, while I also own the Canopus ADVC300 and it does a great job with Hi8 tapes, it only encodes to 8-bit 420 DV-AVI. I recommend this for archiving, but not for heavy restoration. But I don't recommend 10-bit raw video either. A better and cheaper workflow is to purchase a USB converter for around $40 USD. The one I own is the i-o data gv-usb2. It has S-Video-In and with AmarecTV captures to uncompressed 8-bit 422 video which is ideal for heavy restoration. So believe it or not, the best capture workflow for Hi8 S-Video is a $40 USB capture device and free software. No need for TBCs or other expensive capture gear.

Striving for something like 10-bit raw caps is a complete waste of time and money. There is no sensor data to play with to apply your own debayering algos for challenging moire or custom LUTs for HDR grades. Home VHS/Hi8 video will never look like HD/UHD video we have become accustomed to. A better way to approach such a project is to recognize that home video has a certain aesthetic and try to enhance the video with that in mind versus striving to achieve a modern HD/UHD look.

As for the rest of the workflow, I would recommend de-interlacing with QTGMC. It is arguably the best de-interlacer, but it is slow. So I would also invest in setting it up for multi-thread. For this, Avisynth works fine, and there is no need to use VS unless you are already fluent with VS. QTGMC will apply some noise reduction in the process, so you will have to decide if you want to do any further NR beyond that. IMHO, less is always more with NR. At this point, my advice with the standard free tools stops as I prefer full fledged NLEs for any work beyond "simple" capturing and de-interlacing.

[HuBanfiT]

Thank you, and keep the suggestions coming.

johnmeyer,

Thank you. I'm half a globe away from any Costco or Walmart. Although we do have archival services over here as well - I'll try to look into them again to see whether they improved. They used to be plain direct S-Video -> HW MPEG-2 conversion shops, and that is not a route I want to subject my material to, as it is particularly sensitive to bad motion estimation - there are quick and sudden movements in the fights sometimes, which could mean half a screen of movement in a few frames when the camera/tripod was positioned right next to the fighting area. Now this amount of complexity (change in picture content) I doubt those solutions could (have) handle(d).

As for your suggestion for the ADVC300, it goes through DV compression which is considerably more quality reduction that I feel I (or the material) could tolerate, especially as in will be further compressed/encoded and not just edited and played back out to analog, so I seek a more pristine/clean path than going through DV.

SameSelf,

I will look at the USB converter you suggest; I have one of the cheap Easycap ones (with S-Video in - as in: not just composite video), and I batch captured all my tapes with it into lossless 8-bit 4:2:2 video, and aside from never quite having been able to get the preamp levels right, it somehow lacks in picture detail compared to what I used to have before with PCI cards.

Aside from that, I have also found that the horizontal line jitter further throws deinterlacing and noise reduction off enough into oversmoothing land that I would like to fix that issue as well.

And finally, there are some tape issues as described above, which caused it to slightly fall out of sync either on just chroma, but also on the luma as well on about a quarter of the tapes. All this I wish to avoid by a better solution.

Do you think the USB capture device you are suggesting would be better in these respects?

Other than storage requirements (and increased processing load) why would you not suggest 10 bit raw video? No sensor data perhaps, and not enough data on the tape either - however 8 bit is going to be the mimimum bitdepth of the final output material, so if our input is 8 bit already, we will have already added quantization noise on par with our output precision, which is never a good thing.

I do agree it won't even look like HD - however what I am currently getting is definitely less than what I previously saw is on the tapes.

QTGMC - I will look at it again, but last time I checked (about 2-5 years ago, can't recall) it was still working in gamma-corrected space, and still used block-matched motion estimation (MVTools/MVTools2), both of which I hope to do better on using VapourSynth. Once you see the quality a high-bitdepth, linear-light, 4:4:4 processing workflow can bring, it seems pointless - and even limiting, visually - to go back to the confines of 8 bit gamma-corrected 4:2:2. In linear light, naive temporal averaging (on steady images) looks perfect; in gamma-corrected space it looks like wet painting. Ever hard your resizes or noise reduction end up looking like a wet painting - there's naive averaging/interpolation performed on gamma-corrected pixel values at fault. Just look at the picture comparison here to gain an appreciation for the quality impairment working in a gamma-corrected space brings to naive implementations: http://int64.org/projects/resamplehq/ . It is my belief that 90-99% of the resizing quality issues people are trying to avoid by using Spline9/16/36 or Lanczos etc. - but never quite succeed, otherwise there would not be multiple methods still in use concurrently - stem from ignoring/being unaware of this fundament. In VapourSynth it is almost trivially feasible to work in linear light - inherently avoiding all this quality impairment even when using filters where the authors weren't aware/never considered this issue.

I think if you switch to a high-bidtepth, linear light workflow, you'll find yourself much less confined/limited as to how much improvement you can make to your material. (Plus VapourSynth gives you built-in threading for free on top of that, and also leaves the way open for massively parallel - read: GPU-based - implementations.

Yes, editing is another matter. My planned workflow is to capture the raw material in as detailed and faithful way as possible, and avoid/correct/clean up as many issues of the imaging chain up until that point; which, among others, includes: shot noise from the sensor, interlacing, colour transfer, head speed variations (line jitter), tape noise, digitization noise (in reverse order). Once the material is clean, then comes editing. (Although even there can be quality issues if you do more than just basic effects/transitions, unless your NLE can work in linear light.)

Sorry, I realize this all doesn't sound terribly positive - I'm just always looking to do one better.

Anyone have any other suggestions?

[johnmeyer]

Thank you. I'm half a globe away from any Costco or Walmart.

I sincerely apologize for forgetting to look at your location. My mistake.

They used to be plain direct S-Video -> HW MPEG-2 conversion shops, and that is not a route I want to subject my material to, as it is particularly sensitive to bad motion estimation - there are quick and sudden movements in the fights sometimes, which could mean half a screen of movement in a few frames when the camera/tripod was positioned right next to the fighting area. Now this amount of complexity (change in picture content) I doubt those solutions could (have) handle(d).

As for your suggestion for the ADVC300, it goes through DV compression which is considerably more quality reduction that I feel I (or the material) could tolerate, especially as in will be further compressed/encoded and not just edited and played back out to analog, so I seek a more pristine/clean path than going through DV.

There are lots of people in this forum that will agree with you. However, I do film and video restoration for a living, and in my experience, Beta, VHS, 8mm, and Hi8 all have such lousy video that whatever minor changes get made converting them to the DV format will never be noticed by 99.9% of the viewers.

DV is probably the easiest video editing format ever invented, and the ease of editing will make everything go faster and more smoothly. One other advantage of DV is that I have never dropped a frame or had a glitch when capturing DV video. I cannot say the same about my Hauppauge or ATI capture products, and if you look at various capturing forums, you will find, literally, thousands of posts about dropped frames. Many of the videos people send to me that have to be fixed have all sorts of issues with dropped frames (usually followed by repeated frames in order to keep the sound synchronized).

However, it is your decision. I always try to give advice which will give you the fewest problems and let you complete your project in the most timely manner. I know of what I speak: I have had several projects take over a year to complete, and I learned from those experiences to pay a LOT of attention to workflow issues.

[JVRaines]

My planned workflow is to capture the raw material in as detailed and faithful way as possible …

Then I recommend playing back on a Hi8 VCR with built-in TBC and no analog noise reduction or sharpening via s-video (Y/C) to the Aja KONA LHe Plus, which is a top-quality converter that supports 10-bit sampling. Use the bundled Aja software to store uncompressed video/audio on a fast disk or solid-state drive.

[HuBanfiT]

Thank you sir. Yes, I understand, and agree that if one is looking to complete projects in a timely manner (e.g. for a living - like you do), what makes most sense is to find an efficient, reliable and smooth workflow. But I am not in that situation. I spent years on making these recordings, they have nostalgic value both to me and my club mates, so I am emotionally invested in trying to make this as good as possible - often beyond reasonably so.

On a more technical note, one day I would like to take a look at some of the footage that goes through your hands, digitize and process it on a more faithful level than the admittedly very practical DV allows, and show you the difference, and see whether you would say no to it. May one day we get around to doing that!

The reason you are not getting dropped frames with DV is that the very short signal path between your DV capture card and whatever device you plugged into it (tape deck I guess?) has negligible noise/errors, and I guess also between tape and digital out, because there is ample amounts of signal headroom and digital error correction built into the system at each step. When administered in the right amounts, these counter-measures are capable of eliminating 100% of errors and introduce no quality degradations.

Not so on analog recordings however, where circiutry predates the feasibility of such advanced processing, and you have the inherent mechanical instabilities and imperfections, tape dropouts, line jitter, noise, etc. show up in their unmasked reality. These, and the less than meticulous signal processing built into most commercial hardware are the inherent reasons for dropped frames and sync issues people are experiencing when attempting to digitize material from analogue media. If your workflow does feed from analogue media, and you are not experiencing these issues through your DV connection, it is not due to the inherent superiority of DV in itself - it is because some other device before your DV cable has already fought and coped with these analogue issues, and cleaned the signal for you; a DV connection is just the icing on your cake there, it is just a convenience feature. But it, in itself, is not the cause of there being no dropped frames; just as much as the red carpet in a hotel lobby is not the cause of there being no homeless persons inside: the portier on the door is, who doesn't allow them in to begin with. So DV is convenient, yes, and it introduces zero dropped frames, yes - but it just means it forces someone else to fight the battle for it if your source material is analog.

Which mine is.

[HuBanfiT]

My planned workflow is to capture the raw material in as detailed and faithful way as possible …

Then I recommend playing back on a Hi8 VCR with built-in TBC and no analog noise reduction or sharpening via s-video (Y/C) to the Aja KONA LHe Plus, which is a top-quality converter that supports 10-bit sampling. Use the bundled Aja software to store uncompressed video/audio on a fast disk or solid-state drive.

Thank you, I am taking a look. Hmm, it's just a grand, not bad. Do we know how good it is on less than studio quality tape material? Oh wait, you suggest a VCR with TBC, I see. Any suggestions on that perhaps?

[JVRaines]

I use a Sony EV-S3000. The TBC eliminates waviness and rolling. These decks also have dynamic noise reduction, which I recommend leaving OFF because you have more power and flexibility in software processing after capture. The Aja card will give you best-of-class sampling of any stabilized source. A much less-expensive choice is the Intensity Pro or Shuttle from BlackMagic Design, but I'm not sure if it offers 10-bit sampling for SD sources.

[SameSelf]

I am not sure I understand your question. It sounds like you are saying in the past you had good quality caps using PCI capture boards, etc., but now you are not achieving a similar quality using the Easycap USB device? I can't really speak for that device versus the i-o data device as I do not own it. It sounds like maybe you have tape issues that have possibly gotten worse over the years since your original caps?

As for the benefits of 444, 10-bit, raw, linear light, and so on, I don't disagree on any of your points. It is just you can't create something from nothing. Once you capture to 422 and deinterlaced using QTGMC, you have solved 90% of the problems with SD video. At that point, any additional processing is better spent on correcting things like white balance and exposure and stylistic adds like more contrast, LUTs, vignettes, etc. All this is best done in a full flight NLE. And if you really have $1000 to spend on a AJA deck, I would punt on that and get a high end monitor that you can calibrate. You would be very surprised at how different everything we are talking about becomes when you move away from small GUIs to a colored managed workflow.

[HuBanfiT]

I use a Sony EV-S3000. The TBC eliminates waviness and rolling.

Thank you, I will take a look. I seem to recall an S9000 being recommended to me. Any idea how the two compare? Let me research them first.

How difficult tapes did you feed successfully through the S3000?

These decks also have dynamic noise reduction, which I recommend leaving OFF because you have more power and flexibility in software processing after capture.

Yes, thank you. Does the TBC go analog-digital-analog on these units, or does it stay analog (delay line)?

The Aja card will give you best-of-class sampling of any stabilized source. A much less-expensive choice is the Intensity Pro or Shuttle from BlackMagic Design, but I'm not sure if it offers 10-bit sampling for SD sources.

One of the Intensities (4K I think?) does uncompressed, I think even 10-bit, and supposedly it has a TBC as well, but skimming the forum here I remember someone mentioning that several people found it is not very effective (haven't looked into that claim yet).

Another person on here has a thread where they tested a number of analog capture alternatives; one of them from China, company/brand Magewell (can't recall the model number), price $300 or $400; but it yet have to see how effective its TBC is.

[HuBanfiT]

I am not sure I understand your question. It sounds like you are saying in the past you had good quality caps using PCI capture boards, etc., but now you are not achieving a similar quality using the Easycap USB device? I can't really speak for that device versus the i-o data device as I do not own it. It sounds like maybe you have tape issues that have possibly gotten worse over the years since your original caps?

I think what I was asking is whether you have any impression whether the USB capture device you recommend is consideraby better than the run of the mill EasyCap USB capture devices. But I'll try to look it up and see.

As for the benefits of 444, 10-bit, raw, linear light, and so on, I don't disagree on any of your points. It is just you can't create something from nothing. Once you capture to 422 and deinterlaced using QTGMC, you have solved 90% of the problems with SD video. At that point, any additional processing is better spent on correcting things like white balance and exposure and stylistic adds like more contrast, LUTs, vignettes, etc. All this is best done in a full flight NLE.

Well, I did try with QTGMC, and I can go into the technical details if you wish why, but I just didn't find it to work very well for what I needed to do. However, VapourSynth did not exist back then, so now that you mention this, maybe I should pick on the old captures with new, improved SW tools first. It just needs some software development work, and I am not sure when I'll have the time to get around to that, while my tapes continue to slowly deteriorate in storage. (That's probably the psychology of why I would rather spend money quickly on stuff that is overkill, to be on the safe side, than take the time to develop restoration software the way I want it to work.)

And if you really have $1000 to spend on a AJA deck, I would punt on that and get a high end monitor that you can calibrate. You would be very surprised at how different everything we are talking about becomes when you move away from small GUIs to a colored managed workflow.

Not sure I know what you mean by "small GUIs". I could agree with you on buying a better monitor - however they don't make CRT monitors anymore, and I'm in the 8% (altered color vision male): any and all digitals I looked at in the last 20 years (mostly TVs and computer monitors up to about $10.000) simply just did and still do not have good colour for me. So I tend to rely on scopes and old CRTs, and mentally calibrate. Having been in prepress for 15 years I have learned to relate my colour "experience" to the colour reproduction characteristics of the particular display, but me not having standard colour vision makes it even more difficult. Of course a considerable part of that is that back when LCDs did not have saturated enough colors, everyone oversaturated their digital material in a (perhaps subconscious or uninformed) effort to compensate, very often ending up bumping colours into the gamut and destroying colour detail (e.g. in skin tones - as the most annoying example). Now I keep seeing these same issues in TV demo material even today. So I am slowly starting to think so many people can't be wrong, so the problem must be in me.

Probably as a corollary to this all, I paid close attention to scopes instead of on-screen colours, and perhaps as a result of this my issues were less with colour, and more with (lack of) sharpness/clarity/detail.

And also - this I think is the bigger problem - the tape issues.

Apologies, this is not a clear answer, I'm just thinking out loud. And I value and am thankful for all of your inputs.

[SameSelf]

Regarding the i-o data gv-usb2. I can only say for the low price of only $40 USD, it is one of the best capture devices out there (and that is not just my opinion, although you have to look beyond this forum to find those opinions). But unless someone here happens to own both and can post some comparisons, I guess there is no way to know for sure which is better. But the main reason I recommend it is because at $40, that is so ridiculously cheap, what do you have to lose? Just buy one and try it out. If you don't like it, you're only out $40 at the worst and whatever you can recoup on ebay at the best.

As for QTGMC, that is too bad if you don't like the results. But there are many different tweaks you can apply, so it may be just a matter of tweaking some of the settings.

Regarding small GUIs, I may have spoken out of turn since your are dealing with SD footage which of course fits nicely on an HD monitor alongside the controls of an NLE. And while scopes are important when doing any sort of color grading/correction, the real problem is when grading Rec.709 footage when all computer monitors operate in sRGB space. Of course, if you are delivering for the web, then sRGB space is fine. But otherwise you really need a monitor that can hold multiple profiles so you can easily switch between sRGB and Rec.709 which is what all modern TVs operate in. I am always reluctant to recommend a color managed workflow on this forum because it costs significant money. You also need a calibration tool and a separate video card to handle the color science. But if you are about the pull the trigger on an $1000 AJA deck, I think your money is much better spent on having a second monitor for grading. But then, if your vision doesn't allow you benefit from a color managed workflow, then my apologies.

[Master Tape]

Sorry to bump the thread up, i'm also looking for capture all my old Hi8 tapes in the best quality possible (within reason)

I've come across the best cameras for capturing in another thread, but they are all NTSC.. What are the best PAL camcorders for capturing Hi8, which gives TBC with S-Video and stereo out?