Current status of ld-decode / vhs-decode (true "backup" of RF signals)

[dellsam34]

I added issue #617 to ld-decode hoping this will get some attention. It will also open up the possibility of software decoding really weird obscure formats where the players cannot be easily RF tapped. Having universal composite decoding, even though it's not s-video, would open up all sorts of doors.

https://github.com/happycube/ld-decode/issues/617

In that D-2 link you posted, isn't the D-2 deck has a SDI digital out where you can just transfer the data to computer using a USB3/PCIe SDI interface? Like Betacam or any other pro digital format?

[Titan_91]

I know it has some sort of digital link for dubbing losslessly between D-2 machines. I think it's proprietary though, as SDI wasn't standardized until 1 year later in 1989. I also cant find any D-2 documentation that references SDI. But if you were to either build an FPGA to latch onto and dump the digital signal or use some other form of very fast ADC and dump the bits in software, you would get your video signal and your 4 tracks of "better than CD" (whatever that means) PCM audio.

Imagine a D-2 video source captured as a bitstream file and the resulting digitally sampled composite signal decoded using ld-decode! It would be the ultimate archive solution since you would have both the digitally sampled master signal and a fantastic decode of the image. All with no hardware level distortions involved, it would be an all digital workflow.

[dellsam34]

Oh yeah I forgot about SDI being late to the scene, I think it was first developed and standardized in 1989 by SMPTE in the D-1 standard but wasn't used until Sony came up with digital Betacam.

[Cornucopia]

Also, remember, while D1 is component, D2 is composite, so you would still have to separate your luma & chroma.


Scott

[Titan_91]

I have another standalone VCR but I don't think it tracks as well. I'm going to compare the composite outputs of this one and my main machine.

[Titan_91]

This is a Philips model from 2004 I think, even cheaper than my Zenith 2003 DVD/VHS combo. Capturing composite from both. The bandwidth and resolution are worse, especially looking at the patterns on the floor in the same scene. The chroma saturation is also lower. This is probably due to noise reduction or the lower cost of the player, which does reduce the noise on the image at the cost of bandwidth/sharpness. The Zenith model was probably designed for better quality, as is has a DVD to VHS copy feature. I had to stitch this Philips example together in Gimp because the frame wasn't aligned in Rawsource.

Zenith:



Philips:



Audacity comparisons of Zenith and Philips respectively, Zenith on top/left and Philips on bottom/right:







I also tweaked Zcooger's GNU Radio flow chart for NTSC framing. His chart is here:

https://forum.videohelp.com/threads/394168-Current-status-of-ld-decode-vhs-decode-(tru...10#post2588429

My adjustments change input width to 910, input height to 525, display width to 1400, and display height to 1050.

[Zcooger]

Speaking of my VHS-Decode tests I began to make I share some examples from different machines to compare the output with software solution. I also provide reference baseband HackTV signal that has been recorded with Panasonic NV-HD630 and played back by it and also LG DVC5930 and LG V280:
https://forum.videohelp.com/threads/394168-Current-status-of-ld-decode-vhs-decode-(tru...13#post2606582

Reference PAL encoder:
https://open.lbry.com/@Zcooger:1/DVE_CVBS:b?r=8CHypuz7Dp743zh8cYmwe4g8eC9soFfU

Panasonic NV-HD630 standard capture without any device in between and the following samples too:
https://open.lbry.com/@Zcooger:1/Panasonic-NV-HD630-VHS-Playback-Test-06.01.2021:5?r=8...mwe4g8eC9soFfU

LG DVC5930:
https://open.lbry.com/@Zcooger:1/LG-DVC5930-VHS-Playback-Test-06.01.2021:0?r=8CHypuz7D...mwe4g8eC9soFfU

LG V280:
https://open.lbry.com/@Zcooger:1/LG-V280-Playback-Test-06.01.2021:2?r=8CHypuz7Dp743zh8...mwe4g8eC9soFfU

So these are most common playback machines among consumer segment while personally placing the Panasonic a bit higher (non-lazy mechanism and electronical solution).
To be clear, same CX2388X chipset card has been used for standard capture with VirtualDub2 under Windows and CXADC sampling under Linux.

[oln]

This is a Philips model from 2004 I think, even cheaper than my Zenith 2003 DVD/VHS combo.

Hm, I don't know the model but afaik the late year Philips VCRs in the US were typically Funai units? The Zenith XBV342 looks like it has an LG mech. They may actually be using the same Sanyo video chips, but maybe the funai one has worse components and/or is configured differently. I can't find the datasheet for the particular one in the zenith SM, but these late model video chips from Sanyo and others have adjustable sharpness and noise reduction levels that is set digitally over I2C.


I added some code to ld-chroma-decoder to decode NTSC taking burst phase into account, so NTSC color decoding now works-ish. I think it could maybe use some hue control though. The color looked relatively close on the test patterns (but may need some hue tweaking), I wasn't 100% sure about the other ntsc samples I have as I'm not certain about what the original should look like on all of them.

[Attachment 57300 - Click to enlarge]

I also adjusted the NTSC rf filters to reduce the black/white streaks on the samples I have (EDIT: Seems it made one of the samples that was from another VCR worse argh). This did end up with them looking a bit more grainy (though that may be correct, I don't know). There is still something a bit off as I get the same slight smearing on sharp bright to dark transitions as with my PAL samples tho.

The changes are pushed to git, I've now merged everything back to the main vhs_decode branch so that's the one that should be followed/used (

git checkout vhs_decode

after doing git pull if you are on the ntsc_color_broken branch.

[Titan_91]

Awesome update! Is that the old NTSC color bars sample I uploaded a few months ago? Or a different sample?

[oln]

It's your upload yeah.

[Titan_91]

I'm surprised it looks that good. My input gain was maxed out at 31 on the card since I didn't have an amplifier yet, so higher level of noise. That's not a prerecorded tape, so looks like some chroma dots made it past the luma filtering in my VCR during the recording process. I wonder how that retail Tom and Jerry sample would look now. I'm not at my PC at the moment so I'll post a screenshot of that next, unless you beat me to it.

[themaster1]

[QUOTE=oln;2610295]

I added some code to ld-chroma-decoder to decode NTSC taking burst phase into account, so NTSC color decoding now works-ish. I think it could maybe use some hue control though. The color looked relatively close on the test patterns (but may need some hue tweaking), I wasn't 100% sure about the other ntsc samples I have as I'm not certain about what the original should look like on all of them.

[Attachment 57300 - Click to enlarge]

If you get the colors bars right on your tape chances are high for getting "real footage" right assuming the colors bars & footage are on the same tape.
I've experimented before with color bars with my vcrs.

Ideally they look like this (ideally)

[Attachment 57310 - Click to enlarge]

LoadPlugin("C:\Program Files (x86)\AviSynth 2.6\plugins\VScope12\Release\VScope.dll")
import("C:\Program Files (x86)\AviSynth 2.6\plugins\HistogramCMYK.avsi")
####
Colorbars(width=720,height=480,pixel_type="YUY2")
ColorYUV(cont_u=85, cont_v=85)
############ ANALYSYS #######################
ColorYUV(analyze=true)
#Histogram("levels") # A
VideoScope("both", true, "Y") # Luma levels
VideoScope("side", true, "UV") # Chroma levels

Original smpte Cbars captured with my vcr (vcr has no line tbc, no chroma/luma filtering, very raw)
https://mega.nz/file/6Zp2UBIY#6iMTUY5tHawtaSfVtPtx1I0-AkbUm5Ep49f727X-RMs
other smpte: https://mega.nz/file/mJ50zDYY#5MJehROPZ_MJC8j3BvORfGIOi0ecoO1XZcvRuvF1DAM

[oln]

Yeah afaik recordings in studios and such would often have color bars/test cards at the beginning for calibration.

[Titan_91]

Hey, the new NTSC results are pretty good! I also boosted the chroma gain to 1.5 in ld-analyse. I composited luma and chroma again in Gimp and can clearly see more bandwidth and sharpness are recovered from the RF capture compared to my previous composite video one on the same Zenith machine. Look closely at his tail and on-screen text. Current SNR is about 30dB. I need to reduce the noise in my captures, and will test again with my VGA splitter preamp re-connected before the main amplifier. Still some minor smearing to the right as well, but the clipping issue is 100% gone for me.

[Titan_91]

Some more bad news. I re-connected the VGA amp and made sure the capture wasn't clipping. At least my card is not cutting off now and I can adjust the gain from the main amp board as high as needed. Now the decode clips like crazy. I'm also seeing a persistent wiggling pattern when the player is on, but not playing a tape. I don't think the clipping is due to vhs-decode, but my capture. The wiggling pattern is due to a 3.85MHz signal that's always present when the VCR is plugged in, whether or not it is actually on. This signal is surrounded by wide upper and lower sidebands. This of course aliases the 3.4MHz - 4.4MHz luma signal and causes the squiggles, which have always been in my captures.

I'm suspecting this is the switching power supply board, as it sits literally right underneath my input line to my VGA amp. Will need to verify this and try to better shield my VGA amp and cabling, even though I already have ferrite cores on both ends of the VGA amp.

[Titan_91]

That noise is in the RF signal itself, and I managed to clear up the clipping for no apparent reason. I poked around the entire machine to see if that noise was being emitted somewhere else then picked up through induction, but that doesn't seem to be the case. My final guess the head amplifier in this machine being of poor quality. It must be powered all the time whether the VCR is "on" or not. Because the composite output is softer than the vhs-decode RF output, the VCR probably filters out that noise prior to hardware demodulation and processing. Also it seems this new amp board attenuates the 629KHz chroma signal according to my spectrum plots and vhs-decode results. This doesn't seem to affect the actual SNR of the chroma signal though, so color is still recoverable.

[Titan_91]

I think what I'll do next is try to find a dedicated VCR with a functioning s-video output. That way I can check the RF and demodulated outputs. Unless someone already has one and wants to post a luma and chroma sample separately by using vmux=2 and vmux=3 I would guess. I can kind of fudge this by not plugging in the composite cable all the way, but the resulting wireless coupling of the chroma signal is far too weak to use without physically making the connection and getting the combined CVBS signal. Funnily enough the combo player I have has an s-video output and component, but the component out is labeled for DVD use only. The s-video port is not labeled this way, yet there is no signal when playing a VHS tape. It's probably just a firmware setting, but it sucks that someone forgot to turn that on when designing this machine.

[Titan_91]

Update. Forgetting I found the service manual for the Philips machine, I tapped it again and am getting better results. There's a weird chroma jailbars effect coming through on the Tom and Jerry tape, causing some false color banding in the decode. A Bug's Life however, does not have this issue. I think that's just due to differences in the cassettes and how vhs-decode's filters currently handle that, since the filters are statically configured. Also, A Bug's Life has about a 7dB higher overall signal, which helps the chroma come out. Still seems like that is being attenuated by my wideband amplifier, but it also could be the stock Asus Blackbird capture card I'm using.

These captures were done with no 13MHz LPF or 50 ohm termination, using a temporary really long VGA cable going from the VCR to the VGA pre-amp. SNR on Tom and Jerry is 38dB, A Bug's Life is 32dB. Color saturation boosted a bit in post. The RF output of this VCR is definitely stronger than the Zenith/Akai. One of the next things I need to do though is a multiburst recording and playback test comparing RF and composite.

[oln]

I can look into the Tom & Jerry color issue if you are able to upload the sample.

Just got some PAL U-matic samples thanks to Mischa, so will look at adding that as well soon.

I've also merged upstream ld-decode changes, so the color decoders now support yuv output.

[Titan_91]

Thank you. Here's the Tom & Jerry sample from both of my VCRs with my latest VGA preamp + OPA657 amp board. Also included is A Bug's Life from the Philips machine. The Tom & Jerry Philips sample has a paused scene at the beginning and end. Rename the attached .zip file to .tar.gz.

[Titan_91]

Can you upload a small U-Matic sample? Curious to see what that looks like (I know there's no decoding support at the moment).

[Zcooger]

Here's available one for PAL: https://drive.google.com/file/d/1Egegw1wb2Ci7z9UvOuiMIo7PA6XWz2xS/view
And most likely NTSC: https://drive.google.com/drive/folders/1807iYXWgPFfulEA88ZB4F-lrSfZy0wkV

[Titan_91]

Wow, has U-Matic support already been implement in ld-decode/vhs-decode?

[Titan_91]

Ok, more good news! First, I got the best NTSC results overall so far today using the Philips machine. I confirmed the OPA657 amp is attenuating and screwing with the chroma signal. This resulted in the jailbars pattern in Tom & Jerry and streaking in A Bug's Life. These have both been resolved by removing the OPA657 and connecting the VGA amp straight to the card's input. Although I still had to max out the card's gain at 31, I noticed the noise floor for Tom & Jerry without the OPA657 amp is quite a bit lower at about 17dB less just looking at the spectrum of the same tape. And that's with the card's gain maxed out vs. running at like, zero gain with the OPA amp and a much higher signal level coming in to the card. Very surprising. The noise floor reduction with A Bug's Life is only about 11dB, given that tape has a weaker signal. As for actual SNR, A Bug's Life is 35dB and Tom & Jerry is 40dB.

All this was done without the 13MHz filter, and an excessively long input cable still used prior to the VGA amp. So I can probably further improve the SNR more than this by correcting those two things. Exciting! Still needing to boost the gain to see any color with the Gimp compositing method when I really should be using ld-chroma-decoder. Compositing in Gimp is much quicker. Samples attached, rename .zip to .tar.gz.

More decodes:

[Titan_91]

Double post.

[oln]

Wow, has U-Matic support already been implement in ld-decode/vhs-decode?

NTSC (regular-band) U-Matic is supported yeah. The emphasis is probably a bit more off than the VHS one though, I don't really have a good reference to how the output from the u-matic machine should look. I will add in support for (presumably "regular-band") PAL as well since we got some samples of that now. It's is not actually that complicated to add support for it when we already have VHS working, U-matic is pretty similar. Luma works the same way, just with the rf frequencies and deemphasis bands/filters using different constants. Color is simpler than VHS, it's downconvetred in a similar way, but there is no extra phase rotation or inversion like in VHS and other later formats. Or, at least it was that way with regular-band NTSC. I don't know for certain if all the variants are similary simple, but I think most of them are.

SVHS and Hi8 has some additional more complicated filtering and sub-emphasis on the luma signal compared to VHS and normal video8, but I haven't seen anything indicating that the hi-band variants of U-Matic added extra complexity. They just increased the carrier frequencies and widened the bandwidth of modulated frequencies to increase detail. Unlike the consumer formats they also upped the color carrier frequencies on hi-band u-matic to increase color resolution.

If you don't want to merge stuff manually, you can use the gen_chroma_vid_ntsc.sh (and pal equivialent), which will do it for you, provided ld-chroma-decoder and ld-dropout-correct is in PATH. E.g for doing everything in one go:

./vhs-decode -n --cxadc --doDOD -t 4 /path/to/video/file.r8 output_name ; ./gen_chroma_vid_ntsc.sh output_name

It will give you a merged output_name.mkv file

I may have to up the default chroma gain a little. As you note I think it's too low on both PAL and NTSC at the moment.

[oln]

Ah, also, if using ld-analyse, you need to check the "phase compensating decoder" in the chroma decoder options. It's not used automatically at the moment. The mentioned script will also use the corresponding option in ld-chroma-decoder. Then you should get the correct colors, otherwise it uses the old NTSC decoder that's used for laserdisc. Though, the T&J sample does seem to have a few color spots still (this image also has contrast upped a bit in post and was decoded with wider rf filter than default):

[Zcooger]

To

ld-chroma decoder -f ntsc2d

line in gen_chroma_vid_ntsc.sh script you can add

--chroma-gain 1.9

argument. That's what I used to make color look "normal".
If you wish you can also denoise luminance with

--luma-nr 6

by adding it to

ld-chroma-decoder -f mono

command.
Here are the results:

[Titan_91]

Eureka, good find! The mkv files look great, thanks for those as well. I thought the incorrect colors were just due to "NTSC sort of working" and I just thought the color code needed further tweaks. Beyond that, I see two more problems. One, there are slight color fringing/rainbow effects along edges of brightness transitions. Like the hand in the T&J example. The reason for this seems to be the chroma decoder being active when decoding the luma tbc file. I see the same artifacts when using ld-analyse and turning on the chroma option vs. viewing the entire raw field. If a tweak can be made to disable the chroma decoding for the luma.tbc file and only decode chroma for the _chroma.tbc file, that should clear it up.

Two, the color spots are a pulsing signal at even intervals. When looking carefully at a single frame, the spots are diagonal on the image. These are probably due to the poor input cable I'm using which is temporary. It's literally a VGA to RCA adapter cable with unshielded alligator clips attached to the test point.

[Titan_91]

Where can I download gen_chroma_vid_ntsc.sh? I'm having trouble going back through the pages to find it. I then attached the test point directly to the card (still with alligator clips). SNR for T&J increased from 40dB to 42dB. Results with ABL have also been consistent with my tweaks, so I didn't test it this time around. There was also still a very faint broad squiggle pattern that now seems to be gone. Most excellent. But the faint pulses in the chroma signal are still there, and seem to be more prominent without the VGA amp. Hopefully that's just due to the long cable being used, which I can isolate.